Internal Developer Platforms – Part 2: What is an IPD?

What is an Internal Developer Platform (IDP)?

After having introduced Internal Developer Platform in our first blog post of this series, we want to talk about the features of an IDP in more details and write why IDPs matters for companies within a cloud transformation.

First, the concept of an IDP basically consists of three main components:

Firstly, „Internal“ signifies that the platform is designed for use exclusively within an organization. Unlike public or open-source tools, an IDP is tailored to meet the specific requirements and security standards of the company. This focus ensures that internal workflows, data, and processes remain secure and optimized for the company’s unique ecosystem.

Secondly, the „Developer“ component highlights that the primary users of these platforms are the application developers within the organization. The platform is designed with the needs of developers in mind, aiming to streamline their workflows, improve efficiency, and enhance productivity. By centralizing tools and resources, an IDP reduces the complexity developers face, allowing them to focus more on coding and less on administrative or setup tasks.

Thirdly, „Platform“ denotes that the IDP serves as a foundational framework combining various development, deployment, and operational tools into a cohesive environment. This includes integration with version control systems, continuous integration (CI) tools, GitOps practices, databases, and container technologies. By bringing these elements together, the platform facilitates a more streamlined and coherent development lifecycle.

The main objective of an IDP is to simplify and enhance the developer experience by automating processes, centralizing resources, and eliminating unnecessary manual tasks. This includes enabling developers to request resources, initiate pre-provisioned environments, deploy applications, and manage deployments with greater ease and efficiency. As a result, the deployment process, in addition to development, becomes part of the developer’s realm, increasing control and agility.

An IDP typically requires only a coding environment, the Git tool for version control and merging, and the platform itself. This consolidation reduces the need for external websites or the creation of superfluous scripts to execute certain processes or actions, thereby streamlining the entire application development process.

Internal Developer Platforms are generally developed by a dedicated in-house team, known as the Platform Developer Team, which ensures that the platform is customized to fit the company’s needs and goals. However, for companies lacking the resources or expertise to develop their own IDP, there are Platform-as-a-Service (PaaS) options available, providing a complete, out-of-the-box solution from various vendors.

In contrast to IDPs, the term „Internal Developer Portal“ is occasionally mentioned in literature. While it can be used interchangeably with IDP in some contexts, most sources differentiate between the two. The Internal Developer Portal is typically understood as the graphical user interface (GUI) of the IDP, through which developers and sometimes automated systems interact with the platform’s tools and services. This interface simplifies the user experience, making the platform’s functionality more accessible and intuitive.

The Importance of Self-Service

The concept of self-service is a crucial aspect of Internal Developer Platforms (IDP) and significantly enhances their value and utility for developers. Self-service mechanisms within an IDP empower developers by giving them the autonomy to access resources, tools, and environments directly, without needing to wait for approval or intervention from IT operations or other departments. This approach accelerates workflows, promotes efficiency, and reduces bottlenecks in the development cycle.

In a self-service oriented IDP, developers can perform a variety of actions independently. For example, they can request and allocate computational resources, initiate pre-configured environments, deploy applications, and set up automated deployments. Additionally, they can manage scaling, monitor performance, and if necessary, roll back to previous versions of their applications without external assistance. This autonomy not only speeds up the development process but also encourages experimentation and innovation as developers can try out new ideas and solutions quickly and easily.

The self-service capability is underpinned by a user-friendly interface, typically part of the Internal Developer Portal, which simplifies complex operations and makes them accessible to developers of varying skill levels. By abstracting the underlying complexities and automating repetitive tasks, the IDP allows developers to focus more on their core activities, such as coding and problem-solving, rather than on infrastructure management.

Moreover, self-service in IDPs is often governed by predefined policies and templates to ensure that while developers have the freedom to access and deploy resources, they do so in a manner that is secure, compliant, and aligned with the company’s standards and practices. This balance between autonomy and control helps maintain the organization’s operational integrity while enabling the agility required in modern software development.

In summary, self-service is a key feature of Internal Developer Platforms that transforms the developer experience by providing direct access to tools and resources, thereby streamlining the development process, fostering independence, and enabling a more agile and responsive development cycle.

The following table summarizes these elements

FeatureDescription
Developer AutonomyDevelopers can independently access resources, tools, and environments, eliminating the need for IT operations or other departments‘ intervention.
Speed and EfficiencySelf-service capabilities accelerate workflows and reduce bottlenecks, enabling faster development cycles and promoting efficiency.  
Innovation and ExperimentationEmpowers developers to quickly try new ideas and solutions without prolonged setups or approvals, fostering a culture of innovation.           
User-Friendly InterfaceTypically part of the Internal Developer Portal, it simplifies complex operations, making them accessible and manageable for developers of all skill levels
GovernanceWhile offering freedom, self-service is governed by predefined policies and templates to ensure security, compliance, and adherence to company standards

That’s for this post. In the next post, we will talk about the internal components of an IDP.

Platform Engineering for Cloud-Native Organizations

Motivation & Introduction

Within the last years, enterprises have already migrated large portions of their workload to the cloud – whether it is a private, public or hybrid cloud. However, many companies still fail to grasp all the benefits of cloud computing. According to the State of DevOps report released by Puppet and DORA (DevOps Research & Assessment), that highlights a DevOps maturity model, the overwhelming majority of companies are struggling with to reach the highest level of DevOps majority. This results in a set of problems to be solved: First, long lead times can be observed. Secondly, processes are often carried out manually, usually initiated by means of tickets created by developers. Third, the overall complexity of the tools and the way how to integrate them is usually high leading to the fact that developers are overwhelmed. Last, waiting times slow done developers as many companies lack an internal self-service developer platform.

In mature DevOps organizations, the structure typically includes stream-aligned application teams and platform teams. Stream-aligned teams focus on developing and deploying code, while platform teams support them by providing various services and support:

  • Platform servicing such as CI/CD and infrastructure provisioning by using industry-wide best practices such as Infrastructure-as-Code (IaC).
  • Evangelization and mentoring DevOps practices for promoting cultural values, such as communication, transparency and knowledge sharing.
  • Rotary human resources, which means that platform teams might help and provide product teams with human resources when these teams lack of specific skills to accomplish their work.

The post discusses the formation of a platform team to efficiently support enterprises with cloud-native technologies. It explores how this team integrates with product teams and outlines its responsibilities, emphasizing technical requirements like Infrastructure-as-Code and GitOps. The paper’s structure includes a review of related work, a technical background overview, detailing efficient platform team requirements and processes, and concluding with future work considerations.

Related Work

Platform Teams and Platform Engineering emerged as a relatively new trend, with initial publications dating back to 2020. Leite et al. (2020) define platform teams as structures for continuous delivery and discuss their role in organizations and interactions with product teams. Srivastava (2023) emphasizes the importance of Platform and Site Reliability Engineering for both startups and larger organizations, enabling efficient delivery of high-quality, secure, compliant, robust, and reliable products. Seremet et al. (2022) explain the symbiotic relationship between Platform Engineering and Site Reliability Engineering, highlighting shared principles and differences. Puppet’s State of DevOps Report (2023) underscores the significance of platform engineering in achieving DevOps success at scale, its rising prominence, and the benefits it brings to organizations when executed effectively.

Background

Platform Teams in cloud-native environments leverage software engineering principles to accelerate software delivery. Infrastructure-as-Code (IaC) is a key practice, involving declarative descriptions of the cloud environment in source control systems to efficiently provision and manage resources, ensuring consistency and cost mitigation. GitOps extends IaC to all deployments, maintaining a single source of truth for declarative deployment information and utilizing monitoring tools for continuous state reconciliation.

Site Reliability Engineering (SRE) automates IT infrastructure tasks, prioritizing the reliability of scalable software systems amidst frequent updates. SRE emphasizes observability through Service Level Objectives (SLOs) and Service Level Indicators (SLIs) to measure reliability and set availability goals. It also promotes a cultural shift where every failure is viewed as a system reliability issue.

Platform Team Engineering For Cloud-Native-Environments

In organizations dealing with cloud and Kubernetes environments, DevOps teams often face challenges in infrastructure provisioning and security. Platform teams alleviate these issues by abstracting infrastructure concerns from application teams. Companies may not have the size or expertise to form their own platform team, leading to the option of offering platform engineering as a service.

Standardization is crucial for such services to enhance efficiency in supporting application teams. Tooling is required to enable the consumption of Infrastructure-as-Code (IaC) modules and provide self-service capabilities. To facilitate GitOps and security best practices, a well-defined structure in a Git repository is proposed, allowing infrastructure and application deployments to work seamlessly.

Figure 1 illustrates the responsibilities and interaction of the Platform Engineering team, which acts as an abstraction layer over the underlying infrastructure (typically a public or private cloud). It offers various services to product teams to enhance efficiency and manage infrastructure effectively.

  • Most important, a DevOps tool with pipeline support in order to run workflows is provided to run workflows.
  • The Platform Team provides standardized modules that are optimized in terms of programming and security best practices and are compliant for the organization.
  • Application Teams need a way to interact with the Platform Team. In most use cases, using Git is sufficient in order to support IaC pipelines and GitOps workflows. For advanced use cases, a self-service catalog or an internal developer platform can be used.
  • Besides infrastructure deployment support, a CI/CD platform should also be provided. Functionality should include at least support for container images builds and app delivery.
  • In order to support SRE, observability should be integrated in the platform tooling and all provisioned resources should have observability support included.
  • Platform tooling should support product teams in the complete DevOps lifecycle – from building, testing to operating.
  • If Application Teams need support, the Platform Team might provide consultancy and human resources to the respective teams.

Figure 1 depicts the relationship between the Platform Tooling and the Applications that are built on top of it.

The platform engineering team serves as a central point of contact for various aspects of the infrastructure and platform. As the team is responsible for the development, deployment and management of the cloud platform, it can act as the main point of contact for other teams within the organization.