Azure Data Factory Video
5 page views
1140 video views
2024, November 06, Wednesday

#064 How to Do Web Scraping in Azure Data Factory

Apresentamos nesse vídeo a técnica de garinpar dados conhecida como web scraping utilizando o Azure Data Factory, extraindo os valores de cotações da bolsa de valores em tabela de página da internet

We will know the steps:

🔹Create LINKED SERVICE, WEB TABLE;
🔹Use Integration Runtime Self-hosted (INTEGRATION RUNTIME SELF-HOSTED);
🔹Set website URL for stock exchange quotes (YAHOO FINANCE);
🔹Create Source Dataset Web Table (DATASET, WEB TABLE, SOURCE);
🔹Set the table index and preview the data (INDEX, PREVIEW DATA);
🔹Create Target Data Set File System (DATASET, FILE SYSTEM, SINK)Set
🔹the file format type (DELIMITEDTEXT);
🔹Create Pipeline with PIPELINE, COPY DATA ACTIVITY;
🔹Map data type of columns (IMPORT SCHEMAS);
🔹Run by Trigger Pipeline (TRIGGER NOW);
🔹View the result of the generated file on the on-premises server (CSV, WINDOWS SERVER).