Hello World to Microsoft Fabric
Microsoft Fabric combines several Microsoft products: Azure Data Factory (data integration), Azure Synapse Analytics (data warehousing and big data analysis), Power BI (business analytics), Collaboration with Azure Machine Learning.
Concepts
Workspace: container for collections of items such as lakehouses, warehouses, and reports, and to create task flows. Can be project-based or environment-based (e.g. dev, test, prod).
Onelake: unified data storage that brings various data patterns together, e.g. data lake, lakehouse, data warehouse, database (transactional storage)
Task flows: this canvas is pretty confusing. At first glance, it is something like data processing workflow. However, it serves as a visual representation of the work in the workspace to data solution architect (e.g. are you using medallion architecture, at each stage, what elements are involved, e.g. from whichd data source you ingest into landing zone, then to which bronze zone, what transformation performed before going to silver zone, etc.). A workspace only has one task flow.
Data pipeline: A data pipeline is a logical grouping of activities that together perform a data ingestion task. Pipelines allow you to manage extract, transform, and load (ETL) activities instead of managing each one individually. It can call lots of activities, e.g. copy, stored procedure, notebook, dataflow, azure ML, function, etc.
Dataflow: focused on data transformation that uses the familiar Power Query interface (powering Power BI) to ingest, transform, and load data into various destinations
Choose between data pipeline, dataflow, spark, difference
SQL database: serves for OLTP database (while SQL DW, Lakehouse are for analytic workload). However, for AI app on operational data, might be useful.
Fabric Copilot: must have an F64 or higher SKU or a P SKU in certain regions (The Azure OpenAI Service used to power Fabric Copilot is currently deployed only in US datacenters (East US, East US2, South Central US, and West US) and one EU datacenter (France Central).)
Get started
Appendix
Licensing
copilot
Medallion architecture and lakehouse
By leveraging landing zone, we can prevent bad-quality data entering into bronze-layer and do basic transformation, e.g. field type, while data should be as raw as possible in bronze layer to audit purpose
Dimension modeling
GPU availability
