The Use of AWS at foodforplanet
New Insights and Future-Proofing with the AWS Data Platform
Summary
foodforplanet GmbH & Co. KG, the in-house e-commerce agency of Erbacher – the food family, was tasked with building a modern cloud data platform for the entire Erbacher Group. After selecting AWS as their preferred cloud provider, b.telligent was chosen as the AWS Advanced Consulting Partner to establish the necessary infrastructure and implement the first use case. The journey began by integrating various e-commerce data sources into the platform, enabling foodforplanet to gain deeper insights into their online activities and deliver value by applying these insights to their online shops.
Erbacher – the food family and foodforplanet GmbH & Co. KG
Based in Kleinheubach, Bavaria, Erbacher – the food family has been producing animal feed, pet food, and cereal-based food for humans since 1941. The group currently employs over 800 people in three countries. Key subsidiaries include Josera Agrar (producer of cattle and pig feed), Josera Petfood (producer of pet food for dogs, cats, and horses), and Farm Champs, which supports livestock farmers in optimizing animal health and productivity. foodforplanet operates multiple international online shops for the group’s pet food brands, Josera and Green Petfood. The company aspires to sustainability by developing innovative, eco-friendly products and striving for climate-neutral processes.
b.telligent – A Trusted Partner for the Data Platform
To build its AWS-based data platform, foodforplanet partnered with b.telligent, an independent consultancy based in Germany and an AWS Advanced Consulting Partner. Specialized in BI, CRM, DWH, Big Data, Data Science, and cloud technologies, b.telligent was tasked with designing the AWS data platform architecture and implementing the initial use cases. This collaboration followed a successful proof-of-concept project that demonstrated the feasibility of building a data platform on AWS.
The Need for a Data Platform
As a traditional mid-sized production company, the food family recognized that sustainability and digitalization are key to securing the company’s future. After establishing a successful online business, the next step was to elevate their data-processing capabilities to harness new and innovative opportunities presented by data utilization. A modern data platform was required—one flexible enough to meet the diverse demands of the group’s businesses and departments. It needed to support the growing requirements for data storage and processing while enabling scalable solutions, from e-commerce to IoT and Industry 4.0 initiatives, essential for modern production.
Opting for AWS to Build a Modern Data Platform
Leading this initiative for the group, foodforplanet chose AWS for its flexibility and openness. AWS offers a broad range of data and analytics services that integrate seamlessly and are priced on a pay-for-use basis. AWS Lake House Architecture was employed to build a data lake using services like Amazon S3, AWS Glue, and AWS Lake Formation. This architecture provides cost-effective storage without capacity limitations. The data lake is complemented by Amazon Redshift, a cloud data warehouse service that integrates with the data lake via Redshift Spectrum, offering powerful SQL-based access to curated data for end users.
Implementation of the First Use Case
The first use case focused on integrating data from various systems and service providers (webshop, tracking, and online marketing) supporting foodforplanet’s online business.
Using the AWS Cloud Development Kit (AWS CDK), the team defined the platform’s components as code, enabling infrastructure deployment across multiple AWS accounts.
The initial architecture of the data platform consists of a data lake created in AWS S3, utilizing AWS Lake Formation and the AWS Glue Data Catalog for metadata and access management, along with AWS Glue ETL for data processing within the data lake. Data ingestion was implemented using custom connectors in AWS Lambda, AWS Glue, and Meltano, an open-source solution for running data pipelines that leverages a system of readily available connectors and data loaders. Amazon Redshift was selected as the data warehouse service, paired with dbt as the SQL-based, in-warehouse transformation framework. dbt is also an open-source project, enabling developers to create transformations with simple SQL SELECT statements while the framework handles the complexities of managing underlying database objects and commands. The architecture also includes several additional services, such as an API Gateway for programmatic access, a Yellowfin BI server running on an EC2 instance and supported by an RDS database for its metadata to deliver reports and dashboards, as well as CloudWatch and CloudTrail for monitoring and logging.
Benefits of the First Use Case
By establishing a unified data model within the platform, foodforplanet streamlined reporting for online activities and created a single source of truth for analysis. Eliminating manual preprocessing allows users to work with data daily, enabling more informed decision-making. This reduces costs related to data preparation and improves understanding of internal operations.
The platform also empowers foodforplanet to enhance customer experiences by personalizing webshops based on insights. Real-time data integration and advanced analytics enable enhanced product recommendations.
Scaling the Data Platform
For Erbacher – the food family, this solution is the first step toward harnessing the potential of AWS cloud data. Soon, the platform will serve not only foodforplanet but other group entities as well. Future use cases include predictive maintenance for production lines, image recognition for quality control, and logic applications—all built on the foundation of this scalable and flexible AWS data platform.
Du möchtest noch mehr über b.telligent und unser Portfolio erfahren?
b.telligent Newsletter
