[VIDEO] Incremental Data Extracts with Qlik using Delta Tags and QVD Segmentation

Following up on my previous post about using the Delta Tags framework for handling incremental loads in QlikView and Qlik Sense, this time I want to share a video with an example that shows this technique in action.

The following 14-min video is taken from one of the lessons in my Lean Data Processing online course. It shows how the Delta Tags technique easily replaces another commonly used approach to incremental loads (based on variables), and demonstrates why the Delta tags approach makes the process more reliable while still being highly efficient and configurable.

Because this video is taken from the course, and is a continuation of a larger exercise, there are references to other lessons of the course. Still, I think the necessary context is there and hope those references aren’t too confusing. Just note that the the video starts-off with an existing incremental script that uses an approach that is based on storing the delta timestamps in variables as part of the script, and this approach is then replaced with the Delta Tags technique.

Another important aspect of this technique that is demonstrated in the video is how the Delta Tags approach works very well with segmented QVDs (a.k.a partitioned QVDs) in an incremental scenario, something that is not as straightforward to implement with other approaches.

In this example, I’m making use of an external script for handling the segmented QVD store operation. I will be sharing this other script in an upcoming post with an accompanying explanation and example as usual.

Let me know what your thoughts are on this technique.