BEGIN:VCALENDAR
VERSION:2.0
PRODID:icalendar-ruby
CALSCALE:GREGORIAN
BEGIN:VEVENT
DTSTAMP:20260720T132718Z
UID:24836270-5634-4bc9-9301-9df1a134f1ed
DTSTART:20260108T120000Z
DTEND:20260108T150000Z
DESCRIPTION:This workshop will build on the half-day workshop ["Building Sc
 alable and Maintainable Data Pipelines with Omnipy (Part 1 - Beginner leve
 l)](https://tess.elixir-europe.org/events/building-scalable-and-maintainab
 le-data-pipelines-with-omnipy-part-1-ed5010da-bcef-4eca-82b2-fc3ab1f49eaf)
  we are holding before lunch.\n\nIn this second workshop\, participants wi
 ll learn how to develop various types of data flows in Omnipy\, including 
 integration with web services. They will make use of the powerful industry
 -developed Prefect orchestration engine to scale up the game and deploy hi
 gh-throughput ETL flows using external compute resources.\n\nThe workshop 
 is divided into three parts:\n\n1. The first part will introduce the sloga
 n "parse\, don't validate" and show how these concepts are implemented in 
 Omnipy. On this background\, we will introduce the three types of data flo
 ws supported by Omnipy: linear\, DAG\, and function flows. We will also\, 
 through hands-on examples\, show how to make use of various job modifiers 
 to power up and customise predefined tasks and flows to construct more com
 plex data flows.\n1. The second part will focus on integrating data flows 
 with web services through REST APIs. We will mainly focus on extracting da
 ta from data sources\, but will also touch upon loading results onto data 
 sinks. Hands-on examples will introduce tasks and flows that allow flatten
 ing of JSON data into relational tabular form for mapping\, and then restr
 ucturing the results back to JSON.\n1. The last part will introduce Omnipy
 's integration with S3-based cloud storage and the Prefect ETL orchestrati
 on library. As a hands-on exercise\, the participant will scale up the dat
 a flow developed in the second part of the workshop by deploying it on an 
 external compute infrastructure\, potentially the Kubernetes-based NIRD To
 olkit from SIGMA2 (if Prefect-integration in NIRD is finalised in time for
  the workshop).
LOCATION:Moltke Moes vei\,  Moltke Moes vei
SUMMARY:Building Scalable and Maintainable Data Pipelines with Omnipy (Part
  2)
URL;VALUE=URI:https://www.ub.uio.no/english/courses-events/events/dsc/dsday
 s/2026/18-building-data-pipelines-with-omnipy-part2-20260108
END:VEVENT
END:VCALENDAR