Data Engineering
Data Engineering refers to the set of practices used to collect, transform, organize, and distribute data so it can be used effectively.
In a context where data often comes from multiple, raw, and unstructured sources, CATIE plays a key role in retrieving, cleaning, transforming, and storing this data to make it accessible and usable.
Cloud – Big Data Architecture
This field focuses on designing secure and scalable software architectures, capable of processing data efficiently regardless of volume.
To achieve this, we rely on services offered by major cloud providers (Scaleway, OVH, Amazon Web Services, Azure, etc.) or on open-source frameworks, in order to build data processing pipelines tailored to each company’s specific use cases and constraints.
Some example applications:
-
Automatic extraction and storage of photos received by email using a serverless approach to minimize costs
-
Deployment of large-scale web scrapers that automatically restart and switch IP addresses in case of errors
-
Selection of a suitable database and data model to ensure fast query performance