{"id":241617,"date":"2023-05-05T17:02:02","date_gmt":"2023-05-06T00:02:02","guid":{"rendered":"https:\/\/virtual-dba.com\/?p=241617"},"modified":"2023-06-29T19:12:33","modified_gmt":"2023-06-30T02:12:33","slug":"change-data-capture-using-azure-data-factory","status":"publish","type":"post","link":"https:\/\/virtual-dba.com\/blog\/change-data-capture-using-azure-data-factory\/","title":{"rendered":"Change Data Capture Using Azure Data Factory"},"content":{"rendered":"\n<p>Capturing changes from the data sources for implementing an incremental data load is a common requirement in data engineering pipelines. One of the cool features of Azure Data Factory is CDC (Change Data Capture). This can be implemented using various options available in mapping data flow or Change Data Capture factory resource.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Mapping Data Flow<\/h2>\n\n\n\n<p>We can graphically design the data flows and transformations using the mapping data flows. Moreover, mapping data flow allows us to model the data transformations without writing code.<\/p>\n\n\n\n<p>To implement Change Data Capture, first, create a new mapping data flow and select the source, as shown in the screenshot below. Once we choose the source dataset, if we go to Source Options, we have the Change Data Capture checkbox, as highlighted in the screenshot below.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"940\" height=\"405\" src=\"https:\/\/virtual-dba.com\/wp-content\/uploads\/1-change-data-capture-using-azure-data-factory-change-data-checkbox.png\" alt=\"Change Data Capture Using Azure Data Factory   Change Data Checkbox\" class=\"wp-image-241614\" srcset=\"https:\/\/virtual-dba.com\/wp-content\/uploads\/1-change-data-capture-using-azure-data-factory-change-data-checkbox.png 940w, https:\/\/virtual-dba.com\/wp-content\/uploads\/1-change-data-capture-using-azure-data-factory-change-data-checkbox-480x207.png 480w\" sizes=\"(min-width: 0px) and (max-width: 480px) 480px, (min-width: 481px) 940px, 100vw\" \/><\/figure>\n\n\n\n<p>We have two options within this. One is an incremental column, and the other is a database-maintained change log-based approach.<\/p>\n\n\n\n<p><strong>Incremental Column:<\/strong><\/p>\n\n\n\n<p>Changed records would be automatically detected by data flows using this incremental Column approach. This applies to files as well. Changed files would be detected using the last modified timestamp. We must select a column that would be used for identifying changes (an incremental column) when we want to extract delta data.<\/p>\n\n\n\n<p>We must select a column that would be used for identifying changes (an incremental column) when we want to extract delta data.<\/p>\n\n\n\n<p>We can observe both options for Change Data Capture from the screenshot below. For incremental data extraction, we have to provide the column name we want to use for identifying changes. As of this writing, Azure Data Factory supports the following sources for this option.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Azure Blob Storage<\/li>\n\n\n\n<li>ADLS Gen2<\/li>\n\n\n\n<li>ADLS Gen1<\/li>\n\n\n\n<li>Azure SQL Database<\/li>\n\n\n\n<li>SQL Server<\/li>\n\n\n\n<li>Azure SQL Managed Instance<\/li>\n\n\n\n<li>Azure Database for MySQL<\/li>\n\n\n\n<li>Azure Database for PostgreSQL<\/li>\n\n\n\n<li>Common data model<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"940\" height=\"402\" src=\"https:\/\/virtual-dba.com\/wp-content\/uploads\/2-change-data-capture-using-azure-data-factory-resources.png\" alt=\"Change Data Capture Using Azure Data Factory  Resources\" class=\"wp-image-241613\" srcset=\"https:\/\/virtual-dba.com\/wp-content\/uploads\/2-change-data-capture-using-azure-data-factory-resources.png 940w, https:\/\/virtual-dba.com\/wp-content\/uploads\/2-change-data-capture-using-azure-data-factory-resources-480x205.png 480w\" sizes=\"(min-width: 0px) and (max-width: 480px) 480px, (min-width: 481px) 940px, 100vw\" \/><\/figure>\n\n\n\n<p><strong>Database-maintained Change Log:<\/strong><\/p>\n\n\n\n<p>Another option for implementing the CDC is using the native change data capture<strong> <\/strong>technology in the databases. This option does not require any column to identify changes. Instead, it uses the internal change log maintained by the database (a log-based CDC at the source database\/table level).<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"878\" height=\"490\" src=\"https:\/\/virtual-dba.com\/wp-content\/uploads\/3-change-data-capture-using-azure-data-factory-database-table-level.png\" alt=\"Change Data Capture Using Azure Data Factory  Database Table Level\" class=\"wp-image-241610\" srcset=\"https:\/\/virtual-dba.com\/wp-content\/uploads\/3-change-data-capture-using-azure-data-factory-database-table-level.png 878w, https:\/\/virtual-dba.com\/wp-content\/uploads\/3-change-data-capture-using-azure-data-factory-database-table-level-480x268.png 480w\" sizes=\"(min-width: 0px) and (max-width: 480px) 480px, (min-width: 481px) 878px, 100vw\" \/><\/figure>\n\n\n\n<p>In this option, we only need to enable the SQL Server CDC and select the target table in sink transformation. Then, automatically changes from the source table would be transmitted to the target table. Azure Data Factory currently supports the following sources for database change log-based CDC.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Azure SQL Database<\/li>\n\n\n\n<li>SQL Server<\/li>\n\n\n\n<li>Azure SQL Managed Instance<\/li>\n\n\n\n<li>Azure Cosmos DB (SQL API)<\/li>\n\n\n\n<li>SAP CDC<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Change Data Capture Factory Resource<\/h2>\n\n\n\n<p>Microsoft added a new feature to Azure Data Factory to capture change data between source and target.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"940\" height=\"374\" src=\"https:\/\/virtual-dba.com\/wp-content\/uploads\/4-change-data-capture-using-azure-data-factory-source-and-target.png\" alt=\"Change Data Capture Using Azure Data Factory Source and Target\" class=\"wp-image-241616\" srcset=\"https:\/\/virtual-dba.com\/wp-content\/uploads\/4-change-data-capture-using-azure-data-factory-source-and-target.png 940w, https:\/\/virtual-dba.com\/wp-content\/uploads\/4-change-data-capture-using-azure-data-factory-source-and-target-480x191.png 480w\" sizes=\"(min-width: 0px) and (max-width: 480px) 480px, (min-width: 481px) 940px, 100vw\" \/><\/figure>\n\n\n\n<p>If we observe, there is a Change Data Capture option below the Pipelines option. All we have to do is click on the above option, and it will navigate to the new CDC page, as shown in the screenshot below.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"940\" height=\"401\" src=\"https:\/\/virtual-dba.com\/wp-content\/uploads\/5-change-data-capture-using-azure-data-factory-cdc-page.png\" alt=\"Change Data Capture Using Azure Data Factory CDC Page\" class=\"wp-image-241612\" srcset=\"https:\/\/virtual-dba.com\/wp-content\/uploads\/5-change-data-capture-using-azure-data-factory-cdc-page.png 940w, https:\/\/virtual-dba.com\/wp-content\/uploads\/5-change-data-capture-using-azure-data-factory-cdc-page-480x205.png 480w\" sizes=\"(min-width: 0px) and (max-width: 480px) 480px, (min-width: 481px) 940px, 100vw\" \/><\/figure>\n\n\n\n<p>After filling in the CDC name and selecting the Source type, we must choose the linked service for connecting to the source. We would see a list of source tables to choose from. But this uses an incremental column to perform Change Data Capture &#8211; a date column from the source table. After selecting the tables and incremental columns, we must click on continue. It would navigate to the target options page. Here we must choose the target database, target linked service, and target tables corresponding to the selected source tables.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"940\" height=\"399\" src=\"https:\/\/virtual-dba.com\/wp-content\/uploads\/6-change-data-capture-using-azure-data-factory-target-options-page.png\" alt=\"Change Data Capture Using Azure Data Factory Target Options Page\" class=\"wp-image-241615\" srcset=\"https:\/\/virtual-dba.com\/wp-content\/uploads\/6-change-data-capture-using-azure-data-factory-target-options-page.png 940w, https:\/\/virtual-dba.com\/wp-content\/uploads\/6-change-data-capture-using-azure-data-factory-target-options-page-480x204.png 480w\" sizes=\"(min-width: 0px) and (max-width: 480px) 480px, (min-width: 481px) 940px, 100vw\" \/><\/figure>\n\n\n\n<p>After that, clicking Continue will take us to the mapping page with a Run option to start CDC flow. Once we start this, changes will automatically be pushed from source to target until we stop that option.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"940\" height=\"392\" src=\"https:\/\/virtual-dba.com\/wp-content\/uploads\/7-change-data-capture-using-azure-data-factory-mapping-page-run-option.png\" alt=\"Change Data Capture Using Azure Data Factory Mapping Page Run Option\" class=\"wp-image-241611\" srcset=\"https:\/\/virtual-dba.com\/wp-content\/uploads\/7-change-data-capture-using-azure-data-factory-mapping-page-run-option.png 940w, https:\/\/virtual-dba.com\/wp-content\/uploads\/7-change-data-capture-using-azure-data-factory-mapping-page-run-option-480x200.png 480w\" sizes=\"(min-width: 0px) and (max-width: 480px) 480px, (min-width: 481px) 940px, 100vw\" \/><\/figure>\n\n\n\n<p><strong>Note<\/strong>: This option is still in preview mode in the Azure Data Factory as of this writing. According to Microsoft, we will be charged only for the duration of the CDC run.<\/p>\n\n\n\n<p>If you have any questions about this blog or need help with Azure Data Engineering, please <a href=\"https:\/\/virtual-dba.com\/contact-us\/\">contact us<\/a>.<\/p>\n\n\n\n<p>This blog was originally published here.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Capturing changes from the data sources for implementing an incremental data load is a common requirement in data engineering pipelines. One of the cool features of Azure Data Factory is CDC (Change Data Capture). This can be implemented using various options available in mapping data flow or Change Data Capture factory resource. Mapping Data Flow [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":241618,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"off","_et_pb_old_content":"","_et_gb_content_width":"","content-type":"","footnotes":""},"categories":[4018,4166,4158],"tags":[3689],"class_list":["post-241617","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure","category-blog","category-microsoft","tag-azure"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.1 (Yoast SEO v27.1.1) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Change Data Capture Using Azure Data Factory - VDBA<\/title>\n<meta name=\"description\" content=\"Azure Data Factory captures changes from sources using Incremental Column, Database log-based CDC, and Change Data Capture factory resource.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Change Data Capture Using Azure Data Factory\" \/>\n<meta property=\"og:description\" content=\"Azure Data Factory captures changes from sources using Incremental Column, Database log-based CDC, and Change Data Capture factory resource.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/\" \/>\n<meta property=\"og:site_name\" content=\"Virtual-DBA Remote DBA Services &amp; Support - Certified Database Experts\" \/>\n<meta property=\"article:published_time\" content=\"2023-05-06T00:02:02+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-06-30T02:12:33+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/virtual-dba.com\/wp-content\/uploads\/Change-Data-Capture-Using-Azure-Data-Factory.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"557\" \/>\n\t<meta property=\"og:image:height\" content=\"291\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"XTIVIA\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@virtual_dba\" \/>\n<meta name=\"twitter:site\" content=\"@virtual_dba\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"XTIVIA\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/virtual-dba.com\/blog\/change-data-capture-using-azure-data-factory\/\"},\"author\":{\"name\":\"XTIVIA\",\"@id\":\"https:\/\/virtual-dba.com\/#\/schema\/person\/2d86f74bed0c3f1b49100f7fdf7d78d1\"},\"headline\":\"Change Data Capture Using Azure Data Factory\",\"datePublished\":\"2023-05-06T00:02:02+00:00\",\"dateModified\":\"2023-06-30T02:12:33+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/virtual-dba.com\/blog\/change-data-capture-using-azure-data-factory\/\"},\"wordCount\":664,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/virtual-dba.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/virtual-dba.com\/wp-content\/uploads\/Change-Data-Capture-Using-Azure-Data-Factory.jpg\",\"keywords\":[\"azure\"],\"articleSection\":[\"Azure\",\"Blog\",\"Microsoft\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/virtual-dba.com\/blog\/change-data-capture-using-azure-data-factory\/\",\"url\":\"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/\",\"name\":\"Change Data Capture Using Azure Data Factory - VDBA\",\"isPartOf\":{\"@id\":\"https:\/\/virtual-dba.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/virtual-dba.com\/wp-content\/uploads\/Change-Data-Capture-Using-Azure-Data-Factory.jpg\",\"datePublished\":\"2023-05-06T00:02:02+00:00\",\"dateModified\":\"2023-06-30T02:12:33+00:00\",\"description\":\"Azure Data Factory captures changes from sources using Incremental Column, Database log-based CDC, and Change Data Capture factory resource.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/#primaryimage\",\"url\":\"https:\/\/virtual-dba.com\/wp-content\/uploads\/Change-Data-Capture-Using-Azure-Data-Factory.jpg\",\"contentUrl\":\"https:\/\/virtual-dba.com\/wp-content\/uploads\/Change-Data-Capture-Using-Azure-Data-Factory.jpg\",\"width\":557,\"height\":291,\"caption\":\"Change Data Capture Using Azure Data Factory\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/virtual-dba.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Change Data Capture Using Azure Data Factory\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/virtual-dba.com\/#website\",\"url\":\"https:\/\/virtual-dba.com\/\",\"name\":\"Virtual-DBA Remote DBA Services &amp; Support - Certified Database Experts\",\"description\":\"Remote Database Administration\",\"publisher\":{\"@id\":\"https:\/\/virtual-dba.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/virtual-dba.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/virtual-dba.com\/#organization\",\"name\":\"Virtual-DBA: Remote DBA | Remote Database Administration\",\"alternateName\":\"Virtual-DBA powered by XTIVIA\",\"url\":\"https:\/\/virtual-dba.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/virtual-dba.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/virtual-dba.com\/wp-content\/uploads\/V-DBA-Database-Services-and-Support-Featured-Logo.jpg\",\"contentUrl\":\"https:\/\/virtual-dba.com\/wp-content\/uploads\/V-DBA-Database-Services-and-Support-Featured-Logo.jpg\",\"width\":557,\"height\":291,\"caption\":\"Virtual-DBA: Remote DBA | Remote Database Administration\"},\"image\":{\"@id\":\"https:\/\/virtual-dba.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/virtual_dba\",\"https:\/\/www.linkedin.com\/showcase\/36220649\/\",\"https:\/\/www.youtube.com\/channel\/UCx3AIeUQ2ziTLKZSJDZ-SEg\"],\"description\":\"Eliminate database downtime and spiraling costs with XTIVIA\u2019s Virtual-DBA. In today\u2019s always-on business world, gaps in 24x7 on-call DBA support, neglected maintenance and security, or a stretched team struggling with overwhelming workloads can lead to costly disruptions and threaten business continuity. XTIVIA\u2019s Virtual-DBA provides the immediate, expert database administration you need, exactly when you need it, ensuring optimal performance, ironclad security, and significant cost savings without the burden of expanding your in-house team. The goal of Virtual-DBA is to provide a cost-effective solution for organizations seeking to optimize the security, management, maintenance, availability, and performance of their critical business systems, whether self-managed or cloud-managed (e.g., AWS RDS, Azure SQL Database). We accomplish this through a comprehensive remote DBA service offering designed specifically to meet the Oracle\u00ae, DB2\u00ae, Informix\u00ae, MySQL\u2122, PostgreSQL\u00ae, MongoDB\u00ae, MariaDB, and Microsoft SQL Server\u00ae, CockroachDB, Databricks, AWS, and Azure needs of our clients.\",\"email\":\"info@xtivia.com\",\"telephone\":\"8886853101\",\"legalName\":\"XTIVIA, Inc\",\"foundingDate\":\"1992-05-01\",\"numberOfEmployees\":{\"@type\":\"QuantitativeValue\",\"minValue\":\"201\",\"maxValue\":\"500\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/virtual-dba.com\/#\/schema\/person\/2d86f74bed0c3f1b49100f7fdf7d78d1\",\"name\":\"XTIVIA\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/virtual-dba.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/0d3648a00e319a37cf8d6d19f762acfbbb4fd0320fd8a6d6b1e64f44a2a6f259?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/0d3648a00e319a37cf8d6d19f762acfbbb4fd0320fd8a6d6b1e64f44a2a6f259?s=96&d=mm&r=g\",\"caption\":\"XTIVIA\"},\"url\":\"https:\/\/virtual-dba.com\/author\/xtivia\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Change Data Capture Using Azure Data Factory - VDBA","description":"Azure Data Factory captures changes from sources using Incremental Column, Database log-based CDC, and Change Data Capture factory resource.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/","og_locale":"en_US","og_type":"article","og_title":"Change Data Capture Using Azure Data Factory","og_description":"Azure Data Factory captures changes from sources using Incremental Column, Database log-based CDC, and Change Data Capture factory resource.","og_url":"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/","og_site_name":"Virtual-DBA Remote DBA Services &amp; Support - Certified Database Experts","article_published_time":"2023-05-06T00:02:02+00:00","article_modified_time":"2023-06-30T02:12:33+00:00","og_image":[{"width":557,"height":291,"url":"https:\/\/virtual-dba.com\/wp-content\/uploads\/Change-Data-Capture-Using-Azure-Data-Factory.jpg","type":"image\/jpeg"}],"author":"XTIVIA","twitter_card":"summary_large_image","twitter_creator":"@virtual_dba","twitter_site":"@virtual_dba","twitter_misc":{"Written by":"XTIVIA","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/#article","isPartOf":{"@id":"https:\/\/virtual-dba.com\/blog\/change-data-capture-using-azure-data-factory\/"},"author":{"name":"XTIVIA","@id":"https:\/\/virtual-dba.com\/#\/schema\/person\/2d86f74bed0c3f1b49100f7fdf7d78d1"},"headline":"Change Data Capture Using Azure Data Factory","datePublished":"2023-05-06T00:02:02+00:00","dateModified":"2023-06-30T02:12:33+00:00","mainEntityOfPage":{"@id":"https:\/\/virtual-dba.com\/blog\/change-data-capture-using-azure-data-factory\/"},"wordCount":664,"commentCount":0,"publisher":{"@id":"https:\/\/virtual-dba.com\/#organization"},"image":{"@id":"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/#primaryimage"},"thumbnailUrl":"https:\/\/virtual-dba.com\/wp-content\/uploads\/Change-Data-Capture-Using-Azure-Data-Factory.jpg","keywords":["azure"],"articleSection":["Azure","Blog","Microsoft"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/virtual-dba.com\/blog\/change-data-capture-using-azure-data-factory\/","url":"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/","name":"Change Data Capture Using Azure Data Factory - VDBA","isPartOf":{"@id":"https:\/\/virtual-dba.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/#primaryimage"},"image":{"@id":"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/#primaryimage"},"thumbnailUrl":"https:\/\/virtual-dba.com\/wp-content\/uploads\/Change-Data-Capture-Using-Azure-Data-Factory.jpg","datePublished":"2023-05-06T00:02:02+00:00","dateModified":"2023-06-30T02:12:33+00:00","description":"Azure Data Factory captures changes from sources using Incremental Column, Database log-based CDC, and Change Data Capture factory resource.","breadcrumb":{"@id":"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/#primaryimage","url":"https:\/\/virtual-dba.com\/wp-content\/uploads\/Change-Data-Capture-Using-Azure-Data-Factory.jpg","contentUrl":"https:\/\/virtual-dba.com\/wp-content\/uploads\/Change-Data-Capture-Using-Azure-Data-Factory.jpg","width":557,"height":291,"caption":"Change Data Capture Using Azure Data Factory"},{"@type":"BreadcrumbList","@id":"https:\/\/www.xtivia.com\/blog\/change-data-capture-using-azure-data-factory\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/virtual-dba.com\/"},{"@type":"ListItem","position":2,"name":"Change Data Capture Using Azure Data Factory"}]},{"@type":"WebSite","@id":"https:\/\/virtual-dba.com\/#website","url":"https:\/\/virtual-dba.com\/","name":"Virtual-DBA Remote DBA Services &amp; Support - Certified Database Experts","description":"Remote Database Administration","publisher":{"@id":"https:\/\/virtual-dba.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/virtual-dba.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/virtual-dba.com\/#organization","name":"Virtual-DBA: Remote DBA | Remote Database Administration","alternateName":"Virtual-DBA powered by XTIVIA","url":"https:\/\/virtual-dba.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/virtual-dba.com\/#\/schema\/logo\/image\/","url":"https:\/\/virtual-dba.com\/wp-content\/uploads\/V-DBA-Database-Services-and-Support-Featured-Logo.jpg","contentUrl":"https:\/\/virtual-dba.com\/wp-content\/uploads\/V-DBA-Database-Services-and-Support-Featured-Logo.jpg","width":557,"height":291,"caption":"Virtual-DBA: Remote DBA | Remote Database Administration"},"image":{"@id":"https:\/\/virtual-dba.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/virtual_dba","https:\/\/www.linkedin.com\/showcase\/36220649\/","https:\/\/www.youtube.com\/channel\/UCx3AIeUQ2ziTLKZSJDZ-SEg"],"description":"Eliminate database downtime and spiraling costs with XTIVIA\u2019s Virtual-DBA. In today\u2019s always-on business world, gaps in 24x7 on-call DBA support, neglected maintenance and security, or a stretched team struggling with overwhelming workloads can lead to costly disruptions and threaten business continuity. XTIVIA\u2019s Virtual-DBA provides the immediate, expert database administration you need, exactly when you need it, ensuring optimal performance, ironclad security, and significant cost savings without the burden of expanding your in-house team. The goal of Virtual-DBA is to provide a cost-effective solution for organizations seeking to optimize the security, management, maintenance, availability, and performance of their critical business systems, whether self-managed or cloud-managed (e.g., AWS RDS, Azure SQL Database). We accomplish this through a comprehensive remote DBA service offering designed specifically to meet the Oracle\u00ae, DB2\u00ae, Informix\u00ae, MySQL\u2122, PostgreSQL\u00ae, MongoDB\u00ae, MariaDB, and Microsoft SQL Server\u00ae, CockroachDB, Databricks, AWS, and Azure needs of our clients.","email":"info@xtivia.com","telephone":"8886853101","legalName":"XTIVIA, Inc","foundingDate":"1992-05-01","numberOfEmployees":{"@type":"QuantitativeValue","minValue":"201","maxValue":"500"}},{"@type":"Person","@id":"https:\/\/virtual-dba.com\/#\/schema\/person\/2d86f74bed0c3f1b49100f7fdf7d78d1","name":"XTIVIA","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/virtual-dba.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/0d3648a00e319a37cf8d6d19f762acfbbb4fd0320fd8a6d6b1e64f44a2a6f259?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/0d3648a00e319a37cf8d6d19f762acfbbb4fd0320fd8a6d6b1e64f44a2a6f259?s=96&d=mm&r=g","caption":"XTIVIA"},"url":"https:\/\/virtual-dba.com\/author\/xtivia\/"}]}},"_links":{"self":[{"href":"https:\/\/virtual-dba.com\/wp-json\/wp\/v2\/posts\/241617","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/virtual-dba.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/virtual-dba.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/virtual-dba.com\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/virtual-dba.com\/wp-json\/wp\/v2\/comments?post=241617"}],"version-history":[{"count":0,"href":"https:\/\/virtual-dba.com\/wp-json\/wp\/v2\/posts\/241617\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/virtual-dba.com\/wp-json\/wp\/v2\/media\/241618"}],"wp:attachment":[{"href":"https:\/\/virtual-dba.com\/wp-json\/wp\/v2\/media?parent=241617"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/virtual-dba.com\/wp-json\/wp\/v2\/categories?post=241617"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/virtual-dba.com\/wp-json\/wp\/v2\/tags?post=241617"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}