Azure Synapse Analytics

Azure Synapse Analytics Now in GA and the Public Preview of Azure Purview

I’m here with some exiting news from Microsoft! Last week at a digital conference, Satya Nadella announced the general availability of Azure Synapse Analytics and the preview of Azure Purview, a unified data governance service. Azure Synapse Analytics has been gaining traction while in preview and adding Azure Purview gives businesses the ability to get the most of out their data and analytics.


Let’s talk about Azure Purview. This is a comprehensive data governance service that helps organizations discover all data across the organization. Demos at the digital conference showcased different ways you can use Purview for governance. Some key things are the ability to go multi-cloud, not only in Azure, but others as well. You can also connect with your on-prem environment and your Azure data assets.

For quite some time, those of us in the data disciplines have worked to inventory all the different aspects of data, like column, database and table names, etc., and put all those pieces into a common repository, often referred to as a data dictionary. Microsoft has been working for years to create a product that would be comprehensive enough to help most people with their governance and compliance needs. We’ve now got this with Azure Purview.

Some key highlights pointed out are:

  • A business glossary – no need to manually build a data dictionary.
  • Automated data classification – allows you to know things like data type (Social Security number for instance). You also have custom options and can schedule future scanning and classification on a routine basis. This way you’re getting continual updates, as opposed to a data dictionary where you get snapshot in time unless you manually update.
  • Cloud-based search facility – gives you the ability to find things quickly and easily across a broad series of data assets.
  • Data lineage and reporting – supports the end to end data lifecycle.
  • Power BI facilities

I feel Azure Purview is a very strong offering. Without it I would have either create my own versions of these pieces or using something like Embarcadero, which I used years ago. Another thing to note is that the experience is very similar to the canvas workspace experience in Azure Synapse Analytics, so if you’ve been working with that, it will feel very familiar.

The next part of Microsoft’s announcement is that Azure Synapse Analytics is now generally available. Azure Synapse Analytics is a limitless analytics service which brings together traditional data warehouse and big data analytics in one offering. It brings these together for a unified experience to ingest, prepare, manage, and serve data for immediate machine learning and BI applications. I, and many of our customers, have been using this great product a lot, so this going GA is surely exciting news.

Some noteworthy things with Azure Synapse Analytics are:

  • A new native cloud distributed SQL engine
  • Deep integration with Spark
  • Flexible query options such as serverless and dedicated
  • Integration with Power BI and machine learning
  • TPC-H benchmark at petabyte scale
  • Native Row Level Security (this is not possible with Amazon Redshift or Google BigQuery)
  • Native ML integration for the citizen data scientist
  • Code management – by that their talking about Azure DevOps as another piece that plays well with it.
  • Power BI integration to Teams which I found to be kind of cool

Again, great announcements with both the general availability of Azure Synapse Analytics and the public preview of Azure Purview. These two products combined empowers teams to remove data silos and leverage all data for analytics and data governance.

Need further help with these or any Azure product or service? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or  [email protected].

Rowland GoslingAzure Synapse Analytics Now in GA and the Public Preview of Azure Purview
Read More

Microsoft Announces the General Availability of Azure Synapse Analytics and the Preview of Azure Purview

There was some exciting news from Microsoft today! At a digital conference, Satya Nadella, CEO of Microsoft, announced the general availability of Azure Synapse Analytics and the preview of Azure Purview, Microsoft’s new unified data governance service.

Why are these announcements so important? Because to thrive, businesses need to harness the power of their data.

One of the best ways to harness the power of your data is to remove data silos. While not a new concept, achieving this has been a constant challenge in the history of data and analytics, as many ecosystems continue to be complex and heterogeneous. Organizations must break down all silos to the get the most out  of data and analytics, in a consolidated, secure, and compliant manner. Azure Synapse Analytics and Azure Purview remove silos by creating a unified data analytics and governance service.

Now that we’ve given you an overview of the announcements, let’s break down the details of each Azure service.

What is Azure Synapse Analytics? Azure Synapse Analytics is a limitless analytics service that brings together traditional data warehousing and big data analytics – into one offering! Azure Synapse brings these two worlds together with a centralized experience to ingest, prepare, manage, and serve data for immediate BI and machine learning applications.

Azure Synapse Analytics Highlights

  • New cloud native distributed SQL engine
  • Deep integration with Spark
  • Flexible service query options – Serverless + Dedicated
  • Power BI + ML integration
  • Azure Synapse Link – Enables real-time data analytics with link to your operational database
  • TPC-H benchmark at PB scale
  • Native Row Level Security – This is not possible with Amazon Redshift, Google BigQuery
  • Citizen Data Scientist capabilities with direct ML integration
  • Code Management – Automation, code sync to dev/master branch, and end to end deployment lifecycle
  • Power BI integration from inside the Synapse service
  • Ability to add Power BI reports into Teams for end users

What is Azure Purview? It is critical to enable these capabilities through a comprehensive data governance solution. An organization that does not know where its data is, does not know what its future will be.  Azure Purview is a unified data governance service that helps organizations achieve a complete understanding of their data. Azure Purview empowers users to discover all data across the business, track lineage of data and create a business glossary wherever it is stored: on-premise, across clouds, in SaaS applications, or in Power BI.

Azure Purview Highlights

  • Key important and rich features: Compliance, Catalog and Data Map
  • Sourcing from on-prem, SaaS, and multi-cloud services
  • Data Map: search/browse data by tech/non-tech domain
  • Business glossary
  • No manual building of data dictionary
  • Data scanning by clicking on a button
  • Automated data classification with custom options
  • Schedule for future scanning and classification
  • Easy cloud based data search
  • Data lineage and reporting
  • Free scanning: On-premise SQL Servers, Power BI Service, data sensitivity labeling for O365 E5 customers

The combination of Azure Synapse Analytics and Azure Purview enables organizations to develop the capabilities needed to empower their teams to leverage all data for analytics and data governance, silo-free.

3CloudMicrosoft Announces the General Availability of Azure Synapse Analytics and the Preview of Azure Purview
Read More

Azure Synapse 101

 

Do you want to learn how to bring together enterprise data warehousing and big data analytics? In a recent webinar, 3Cloud consultant Mike Donnelly, explores the components included with an Azure Synapse Workspace. This presentation goes through all the various pieces of Synapse, what it is and what it can do, as well as why people will be using this more and more as people build modern data warehouse.
When you search Azure Synapse Analytics, you’ll see there is Azure Synapse Analytics (formerly known as Azure SQL DW) which has been around for a bit as a data warehouse solution. But you will also find Azure Synapse Analytics Workspaces Preview. This preview resource is the focus today and many of the pieces it includes will be covered (and demoed), including:
• SQL Pools (formerly known as SQL Data Warehouse)
• Built-in SQL Pool (aka SQL On-Demand/Serverless)
• Azure Data Lake Storage (Gen2)
• Spark Compute Pools (not Azure Databricks)
• Integrate (Azure Data Factory)
If you’re interested in learning more about Azure Synapse Analytics and why it will become the tool of choice for building modern data warehouse, this webinar is for you. Watch the complete webinar below.

Be sure to watch for our free weekly webinars that happen every Tuesday at 11 AM EST. Check out our events calendar on our website https://www.3cloudsolutions.com/resources/events-calendar/

Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or  [email protected].

3CloudAzure Synapse 101
Read More

A Look at Azure Synapse Studio (in Preview)

Azure Synapse, formerly SQL DW, is an analytics service that gives you the ability to query data on your terms and at scale. It brings together enterprise data warehousing and big data analytics. Azure Synapse brings together serverless on-demand or provisioned resources with a unified experience so you can ingest, prepare, manage, and serve data for immediate business intelligence and machine learning needs.

In this post I’ll show you a new Azure Synapse feature in preview called Azure Synapse Studio that I’m excited about. The Studio is a one-stop shop for working with data of any size, but particularly big data. You can ingest data, you can explore and analyze data that you already have in the workspace, as well as visualize data using Power BI.

In my video demo included in this post, I’ll walk you through the Azure Synapse Studio and show you what you can do with it. On the Home Page, you can click on the “New” button and see all you can do such as, create a new SQL script which can be executed against your SQL on-demand or another SQL Database that you have connected to your Studio’s workspace or create a Jupiter notebook which you can run in either a SQL or Spark context, along with creating dataflows, pipelines and Power BI reports.

In my demo I’ll dig into the Studio and show you how to:

  • create a notebook, ingest data, explore your data by either working in a notebook or a pipeline.
  • connect to external data like an Azure Cosmos DB or Azure Data Lake Storage Gen 2 instance.
  • create a new database with the Manage feature in which you can create a new SQL pool or an Apache Spark pool.
  • create Linked Services to connect to your Data Factory pipelines and instances to work within your databases

There are so many great things you can do with the Azure Synapse Studio and I think it will be very helpful to many people from data engineers to data scientists and business analysts, allowing them to work together in a single workspace without having to flip back and forth between various tools. I highly suggest you sign up for this preview feature and give it a try.


Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or  [email protected].

Brian CusterA Look at Azure Synapse Studio (in Preview)
Read More

My List of Top Contributors in the Data/BI Community

As I read the many posts from those in the community who I follow, I am reminded that the community brain trust is much greater than any individual. As a writer and blogger, I’m occasionally compelled to express an original thought or opinion that I think is uniquely my own. However, we work in a world where everything comes from somewhere and there are many contributors who I trust and rely upon for advice and cutting-edge information. This “corner” of my blog is to highlight these community contributions that I find informative.

Paul TurleyMy List of Top Contributors in the Data/BI Community
Read More