#Data backup software

Often data that is ingested into ClickHouse is delivered through some sort of persistent queue, such as Apache Kafka. These are some alternatives that have been used in the past, and that may fit in well in your environment. It is a good idea to use several approaches instead of just one in order to compensate for their various shortcomings.ĬlickHouse stores data on disk, and there are many ways to backup disks. There are a variety of possible approaches with their own pros and cons, which will be discussed below. What works for one gigabyte of data likely won’t work for tens of petabytes.

In order to effectively mitigate possible human errors, you should carefully prepare a strategy for backing up and restoring your data in advance.Įach company has different resources available and business requirements, so there’s no universal solution for ClickHouse backups and restores that will fit every situation. However, these safeguards do not cover all possible cases and can be circumvented. ClickHouse has built-in safeguards to prevent some types of mistakes - for example, by default you can’t just drop tables with a MergeTree-like engine containing more than 50 Gb of data. In many cases mistakes like these will affect all replicas.

#Data backup software

While replication provides protection from hardware failures, it does not protect against human errors: accidental deletion of data, deletion of the wrong table or a table on the wrong cluster, and software bugs that result in incorrect data processing or data corruption.