Vertica in Eon Mode: Revive

This blog post was authored by Soniya Shah.

Overview

An Eon Mode database keeps an updated version of its data and metadata in a communal storage location. After you shut down the database, the data continues to reside in communal storage. When you are ready to use the storage again, you can revive the database and provision new compute nodes. When you revive the database, you do not need to wait for nodes to recover or data to be moved. Your cluster is ready to use as soon as you revive it. For example, suppose you are an IoT company using machine learning for predictive maintenance. You have a hybrid deployment with production on premises and a machine learning model training in the cloud. Because of your business needs, you must retrain their model every night. You train the model on the last 7 days of data. Shutting down and reviving the database saves you from having to reload most of the data (6 days’ worth). Instead, you load only the last 24 hours of data. Hibernation allows you to save compute costs throughout most of the day. However, without shutdown and revive, you would need to reload all the data. You can revive the database by: • Using Admintools • Using Management Console

Using Catalog Sync

When you revive a database, it uses the synced catalog maintained in communal storage. Each node independently uploads checkpoints and txnlogs. The txnlogs sync every 5 minutes, which is the default value. You can change this value by modifying the TxnLogSyncInterval configuration parameter. Checkpoints are synced each time a checkpoint is created. A JSON file contains additional cluster information and is located in communal storage. This file indicates the catalog truncation version and the timestamp of the last sync to communal storage: /metadata/databasename/cluster_config.json The CatalogTruncationVersion is the catalog version to which the cluster will rewind to upon revive. Its value is calculated by the minimum of all maximum sync version of all subscribers to a shard. Catalog sync is on by default. However, you can modify catalog sync in two ways: • To force a catalog sync immediately, run the following: SELECT HURRY_SERVICE('SYSTEM','TxnLogSyncTask'); SELECT SYNC_CATALOG(); • To change the catalog sync frequency, run the following. This example shows a sync of 1 minute: SELECT SET_CONFIG_PARAMETER('TxnLogSyncInterval','1 minute');

Diagnosing Catalog Sync

Let’s take a closer look at what else you can diagnose using catalog sync. You can check the state of catalog sync on each node by running the following: select * from vs_catalog_sync_state; You can check the state of catalog truncation by running the following statement. This shows you what will happen if you revive the cluster, and how far behind you will be: select * from communal_truncation_status; You can check which catalog truncation version the nodes see by running the following: select * from vs_catalog_truncation_status; You can check what was synced to shared storage from each node by running the following: select * from dc_catalog_persistence_events For more information, see Shutting Down and Reviving an Eon Mode Database.