Data Engineering Associate with Databricks Practice Exam

In SQL streaming, what does the "append" output mode do?

1. Overwrites the entire table

2. Resets the output every batch

3. Incrementally adds new data with each batch

4. Disallows any new entries

The correct answer is: Incrementally adds new data with each batch

The "append" output mode in SQL streaming is designed to incrementally add new data with each batch processed. This means that whenever a new batch of data is received, only the newly generated rows are added to the existing dataset or table, rather than modifying or removing any previously stored data. In this mode, as streaming data flows into the system, it is continuously appended to the end of the existing dataset. This is particularly useful in scenarios where maintaining a historical record of incoming events is necessary, and you only want to track the latest additions without altering previously processed records. This mode contrasts with other options like overwriting the entire table, which would replace all existing data, or resetting the output every batch, which would essentially clear out previous results and create a fresh output. The option that disallows any new entries is also inconsistent with what "append" mode is designed to accomplish. Instead, append mode supports ongoing data ingestion and is commonly used in real-time analytics applications where capturing every new piece of information is critical.