As I read the many posts from those in the community who I follow, I am reminded that the community brain trust is much greater than any individual. As a writer and blogger, I’m occasionally compelled to express an original thought or opinion that I think is uniquely my own. However, we work in a world where everything comes from somewhere and there are many contributors who I trust and rely upon for advice and cutting-edge information. This “corner” of my blog is to highlight these community contributions that I find informative.
Azure SQL Data Warehouse (DW) has been improving its capabilities day-by-day. This relational database platform used by Microsoft is known to be faster and cheaper than other major cloud data warehouse solutions.
We all know life can get hectic. Here at Pragmatic Works, we’re no different. But one of our goals is to learn something new about Azure every day, as things are constantly changing and being updated. Many people are still learning all the amazing things they can do within the Azure cloud and we want to help. Our posts in our Azure Every Day series are a great way to learn more about Azure each week.
There are many options for data storage, how do you know which is right for your data? Today I’d like to discuss storage in relation to the architecture of the modern data warehouse and to shed some light on your options.
Sometimes I get so involved in my repeatable processes and project management that I forget to look up. Such is the case of the December 2018 ability to parameterize linked services. I could not rollback and rework all the ADF components this impacted which had already gone to production, but oh hooray! Moving forward, we can now have one linked service per type, one dataset per linked service and one pipeline per ingestion pattern. How sweet is that? Au revoir to the days of one SSIS package per table destination.
This is blog post 3 of 3 on using parameters in Azure Data Factory (ADF). Blog post #1was about parameterizing dates and incremental loads. Blog post #2was about table names and using a single pipeline to stage all tables in a source. Today I am talking about parameterizing linked services.
Disclaimer:Not all linked service types can be parameterized at this time. This feature only applies to eight data stores:
Azure SQL Database
Azure SQL Data Warehouse
Azure Database for MySQL
Concept Explained: The concept is pretty straightforward and if our goal is to have one linked service per type, a parameterized Azure SQL Database linked service might look like this:
What do you know about the database tool from Microsoft called Azure Data Studio? Azure Data Studio is a free Microsoft desktop tool (initially called SQL Operations Studio) that can be used to manage SQL Server databases and cloud-based Azure SQL Databases and Azure SQL Data Warehouse systems.
Today I’d like to share a great resource that I found when setting up a demonstration of Azure SQL Data Warehouse. It’s a tutorial from Microsoft that allows us to very easily load a large sample data set into Azure SQL Data Warehouse for free.
Recently, Pragmatic Works has been on engagements helping our customers to migrate their on-premises databases to the Azure cloud. We’ve been asked to migrate such things as: SQL databases to a SQL database in the cloud; migrations off their APS or PDWs utilizing Azure SQL Data Warehouse; and MySQL and PostgreSQL databases to the cloud.