The group wanted to industrialize its data digital transformation project, aiming to create a global platform to provide business intelligence and analytics to operational data for its global renewable energy fleet.
Based on a recommendation from Amazon Web Services (AWS), the company reached out to Cloudreach for assistance with the delivery of its cloud governance strategy and data lake platform. The goal of this wider project was to provide a global framework to support the design, implementation and operations of the cloud-based infrastructure for the digital platform, with a particular focus on:
- Establishing production, pre-production and development environments built on the AWS platform.
- Put in place the processes and governance necessary to maintain regulatory controls and best practices relating to security, compliance, data sovereignty and network configuration.
- Define roles and responsibilities within the group to support the digital platform.
- Provide a resilient and highly scalable platform.
- Provide training and support to the company’s IT teams to adequately support the digital platform.
- Manage and optimize costs of the digital platform and make recommendations to adopt best practices.
- Encourage innovation and create a path for the industrialization of new ideas.
The energy group selected AWS as its cloud provider as it had a strong preference to utilize Platform-as-a-Service (PaaS) infrastructure wherever possible. Cloudreach was tasked with identifying specific product recommendations on the AWS platform.
Cloudreach’s global reach and 11 years of AWS expertise, made it an ideal candidate to help the group embark on this journey.
The first step was to unify tools and programming language across the customer’s globally-dispersed teams. This helped the teams produce a catalogue of high-quality, secure and well-tested Cloud Formation templates that were deployed in an automated fashion via CodePipelines in all customer environments.
From the very early stages of the design, the group was determined to prioritize the use of microservices, Infrastructure-as-code and AWS managed services. Cloudreach based its entire work on this prerequisite, using its deep expertise to provide an extended agile and reusable amount of code-based infrastructure.
Another challenge was to build a complete developer environment based on AWS Managed Services only, that would follow security and compliance regulations from the energy sector. This involved automated provisioning of user workspaces, integration with CodeCommit and CodeBuild, and enrolling all deployed VMs into AWS Systems Manager. This level of automation would not be possible without extensive use of Lambda, Step Functions and Cloudwatch Events.
The team deployed the governance solution spanning four AWS accounts and three regions. Infrastructure-as-code practice enabled the team to onboard new AWS accounts with the required setup in a matter of minutes within the highly regulated environment..
Thanks to the rich offerings of AWS, the group could incorporate different ingestion mechanisms tailored to its specific requirements autonomously. Cloudreach helped deploy and manage forward the solutions that involved streaming data via Kinesis Firehose to Redshift, managing data transformation in AWS Glue, integrating web apps with Cognito and CloudFront and performing self-managed enrollment of IoT devices.
The Partnership with Cloudreach
Cloudreach took a personal, customized approach to the engagement, placing highly talented engineers and architects together with the customer’s development teams. This collaboration enabled a smooth transition to the new cloud environment while providing the full potential of a serverless-first approach. It also helped to overcome smaller, everyday challenges, like troubleshooting third-party software well past its supported lifecycle and improving Lambda utilization in a shared environment.
Cloudreach engineers and the customer’s teams grew together in knowledge and experience, improving deployed solutions over time which resulted in lower costs, simplified architecture, and better maintainability.
In addition to the technical aspect, Cloudreach helped the group create and transition into a Cloud Operating Model, forming joint teams to support the company’s software and provide agility and cloud expertise.
The Data Lake and overall journey to the cloud allowed the organization to break down all existing silos between their various technical teams. Thanks to a centralized, documented and highly scalable platform, all teams are now integrated. This platform also delivers agility, independence and autonomy across the organization, enabling teams to take ownership of projects. The developers can now develop and test products immediately on the platform where, in the past, they were constrained to hand over this critical part of their work to IT.