Internal Developer Platforms – Part 9: Backstage Kubernetes Deployment

In the last blog post, we talked a lot of the installation and the architecture of Backstage. We also gave some design considerations for the Kubernetes installation. Now it is time to come up with some details for the Kubernetes installation. 

We will use our design from the last blog post as a basis:

A deployment flowchart for a Kubernetes setup used in Spotify Backstage. The process begins with the Internet cloud icon, connected to a yellow rectangle representing an ingress resource labeled 'Ingress: backstage-ingress'. This ingress directs traffic to two services within the cluster. The first service, outlined in a red dashed rectangle, is labeled 'Service: backstage' of 'Type: ClusterIP', indicating it's internally accessible within the cluster. It routes to a blue dotted rectangle symbolizing a pod, with a green rectangle inside denoting the 'container: backstage'. This container is part of the 'Deployment: backstage'. An arrow points from the backstage service to another service for the Postgres database, also a red dashed rectangle labeled 'Service: postgres' of the same type. It connects to another pod, indicated by a blue dotted rectangle containing a 'container: postgres', above which sits a green cylinder representing the 'Postgres database'. This setup within the pod is part of the 'Deployment: postgres', highlighting the backend database connectivity for the application.

In the following, we will use a namespace called “backstage” for the installation. If you don’t have it so far, simply create it with

kubectl create ns backstage

Postgres Database

Let’s begin with the Kubernetes Deployment for the Postgres database. We need the following components:

  • A Persistent Volume Claim for storing the data
  • A Kubernetes Secret to store the credentials
  • The Postgres Deployment itself 
  • A service manifest to expose the deployment

Lets begin with the Persistent Volume Claim:

# kubernetes/postgres-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pvc
  namespace: backstage
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2G

Now continue with the Secrets manifest. Remember that the data is base64 encoded. So if our user is “backstage”, the password is “test” and the port is 5432, the corresponding YAML looks like the following:

# kubernetes/postgres-secrets.yaml
apiVersion: v1
kind: Secret
metadata:
  name: postgres-secrets
  namespace: backstage
type: Opaque
data:
  POSTGRES_USER: YmFja3N0YWdl
  POSTGRES_PASSWORD: dGVzdA==
  POSTGRES_SERVICE_PORT: NTQzMgo=

Now let’s continue with the deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
  namespace: backstage
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:13.2-alpine
        imagePullPolicy: 'IfNotPresent'
        ports:
        - containerPort: 5432
        envFrom:
        - secretRef:
            name: postgres-secrets
      volumes:
      - name: postgresdb
        persistentVolumeClaim:
          claimName: postgres-pvc

Now, we expose the Postgres deployment as a Service:

# kubernetes/postgres-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: postgres
  namespace: backstage
spec:
  selector:
    app: postgres
  ports:
    - port: 5432

Backstage Components

After having the database running in our Kubernetes cluster, we can continue with Backstage installation itself. We need the following components:

  • A Secret manifest for storing credentials. This includes values for Backstage like GitHub Tokens or docker config json files.
  • The Backstage deployment
  • The Backstage service 
  • An ingress

Let’s begin with the secret:

# kubernetes/backstage-secrets.yaml
apiVersion: v1
kind: Secret
metadata:
  name: backstage-secrets
  namespace: backstage
type: Opaque
data:
  GITHUB_TOKEN: Z2hwX0Q3VGfc bayernblablawdsFsSnEwOW44RW1JMm0fcn2OEZrSDfuerthROMaufstiegDdJQg==
---
apiVersion: v1
kind: Secret
metadata:
  name: pull-secrets
  namespace: backstage
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: esadfjlegojksdfalkideenbuchsfd6000sdfjkls25Fd09XNDRSVzFKTW0wzasfdsfdoiURkSlFnPTfs0ifX19

The following manifest depicts the Backstage deployment (remember you need your backstage container image stored in some registry):

# kubernetes/backstage.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: backstage
  namespace: backstage
spec:
  replicas: 1
  selector:
    matchLabels:
      app: backstage
  template:
    metadata:
      labels:
        app: backstage
    spec:
      containers:
      - name: backstage
        image: ghcr.io/my-backstage-project/backstage:0.1
        imagePullPolicy: Always
        ports:
        - name: http
          containerPort: 7007
        envFrom:
        - secretRef:
            name: postgres-secrets
        - secretRef:
            name: backstage-secrets
      imagePullSecrets:
        - name: pull-secrets
 
 
# Uncomment if health checks are enabled in your app:
# https://backstage.io/docs/plugins/observability#health-checks
#          readinessProbe:
#            httpGet:
#              port: 7007
#              path: /healthcheck
#          livenessProbe:
#            httpGet:
#              port: 7007
#              path: /healthcheck

Now let’s expose backstage as a service:

# kubernetes/backstage-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: backstage
  namespace: backstage
spec:
  selector:
    app: backstage
  ports:
    - name: http
      port: 80
      targetPort: http

Finally, we need an Ingress:

# kubernetes/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: backstage
  name: backstage-ingress
spec:
  defaultBackend:
    service:
      name: backstage
      port:
        number: 80

Setting up an Ingress differs a little bit in every Kubernetes distribution. In the following, we use Google Kubernetes Engine and the manifest would automatically create a Google Load Balancer. In addition, we don’t you HTTPs for our deployment. Setting up encryption would need an certificate and some configuration. However, as we want to concentrate on Backstage, we will skip these configuration here.

That’s it for this blog bost. In the next post, we will focus on some Backstage internals.

Internal Developer Platforms – Part 8: Spotify Backstage Architecture and Installation

After having discussed the main use cases in the last blog post, let’s now continue with some internals. In this blog post, we will discuss the architecture and the installation of Backstage.

Architecture

Backstage is structured around three core components: Core, plugins, and the application instance. The „Core“ provides the basic framework and essential functionalities like the UI interface, including elements such as the navigation bar on the left side. The application itself is a configured instance of a Backstage app, maintained by the platform team, integrating functionalities from Core and various plugins into a cohesive unit for production environments.

Plugins are vital, supplying additional functionalities either developed internally or sourced externally. These are divided into frontend and backend plugins. Frontend plugins are typically client-side React applications linked to the Backstage UI and accessible via specific URLs like „/catalog“ for the Software Catalog. These plugins export „Extension Components“ from the /backstage/plugins directory, which integrate into the frontend with routing information and metadata.

Backend plugins handle operational aspects and may incorporate services like caching systems and databases (primarily PostgreSQL or SQLite for production) managed within a Kubernetes cluster. Each plugin may create its logical database for isolation, enhancing the system’s organization and scalability.

The architecture heavily utilizes npm packages to deliver libraries and structure the code, with both frontend and backend systems employing various packages to define plugin boundaries and facilitate their integration. This structure not only enhances the application’s modularity and scalability but also supports future developments and optimizations in the backend system.

The package infrastructure is depicted in the following picture.

An architecture diagram detailing the structure and dependencies of a Backstage application. The diagram is divided into various sections: 'Frontend App Core', 'Frontend Plugin Core', 'Frontend Libraries', 'Plugin Libraries', 'External Plugin Libraries', 'Common Tooling', and 'Common Libraries'. Each section contains packages with namespaces like '@backstage/core-app-api' and '@backstage/app-defaults' for the Frontend App Core, '@backstage/core-plugin-api' and '@backstage/test-utils' for the Frontend Plugin Core, and others like '@backstage/theme' and '@backstage/integration-react' for Frontend Libraries. Plugin Libraries section includes individual plugin identifiers with React, common, and node modules. External Plugin Libraries follow a similar pattern but for plugins not native to the main application. Common tooling and libraries feature CLI tools and shared resources like '@backstage/cli', '@backstage/types', and '@backstage/config'. Lines connect these components to demonstrate their interoperability and how they contribute to the overall functionality of the Backstage ecosystem. Color coding differentiates between frontend packages, isomorphic packages, backend packages, and CLI packages for clarity.

Installation Considerations

In the following we will provide a detailed summary of deploying and setting up a version 1.15 Backstage instance using a Kubernetes cluster. The installation process begins locally, where Backstage can be installed on either a Linux system or through the Windows Subsystem for Linux (WSL). Essential tools required include curl, wget, yarn, git, docker, and the Node Version Manager (nvm). Nvm is used to install Node.js in its current Long-Term Support (LTS) version. The Backstage application, for example named „my-backstage-app,“ is created via the command line using

`npx @backstage/create-app@latest`.

The directory structure generated during this setup includes a `/node_modules` folder for Node.js project modules, and other critical folders and files like `app-config.yaml` and `app-config.production.yaml`, essential for configuration. These YAML files distinguish between test and production settings, allowing tailored configurations for different environments. Notably, `app-config.production.yaml` has the ability to override settings in `app-config.yaml` if similar settings are present.

Once installed, the application can be started locally using

`yarn dev`,

making the frontend accessible at `localhost:3000` and the backend at `localhost:7007`. At this point, the instance includes fundamental features such as the Software Catalog. Developers are free to further customize or extend functionalities from this base.

Integration with GitHub is essential for managing the Software Catalog. It necessitates setting up an authentication system within Backstage, typically configured for OAuth with GitHub. This setup involves registering the application in GitHub to obtain a client ID and secret, and configuring the `app-config.yaml` to use these credentials.

For deployment in a Kubernetes environment, `app-config.yaml` is further configured to specify PostgreSQL as the database, pulling connection details from the Kubernetes environment variables. It also ensures that backend requests are not upgraded from HTTP to HTTPS in the absence of a valid certificate.

The Kubernetes deployment is based on building a Docker image from the Backstage setup. This image is then pushed to a public Docker Registry. The Kubernetes configuration involves a series of YAML files that define the service architecture, including the setup for the PostgreSQL database and Backstage services, and the establishment of an ingress for public access. Deployment begins with the PostgreSQL database to ensure that the necessary database scripts run before the Backstage services start.

Finally, the entire setup within the Kubernetes namespace „backstage“ involves deploying the database, secrets, services, and ingress to ensure that Backstage is fully operational and accessible via a public IP address defined in the ingress configuration. This setup ensures that Backstage is scalable, secure, and ready for production use.

A simplified Kubernetes deployment diagram illustrating the connection from the Internet to a Postgres database within a containerized environment. The flow starts with the Internet symbol, leading to a Kubernetes cluster represented by a dashed yellow rectangle labeled 'Ingress: backstage/ingress'. Inside the cluster, there are two services: the 'Service: backstage' with a red border, connected to the 'container: backstage', and the 'Service: postgres' also with a red border, which is linked to the 'container: postgres'. Each service is of type 'ClusterIP' suggesting internal cluster communication. The 'container: backstage' is encapsulated within a blue-dotted rectangle, indicating a 'Pod', and labeled 'Deployment: backstage'. Similarly, the 'container: postgres' is part of a 'Pod' labeled 'Deployment: postgres', with a green cylinder representing the 'Postgres database'. This diagram effectively demonstrates the path of a request from the internet through the ingress to the services running in a Kubernetes environment.

That’s it for today. In the next blog post, we will come up with some details for the installation.

Internal Developer Platforms – Part 7: Spotify Backstage Core Functionality

As discussed in the last blog post, Backstage already comes with a lot of built-in features. It’s time to discover those functionalities. In detail, we will cover the following:

  • Backstage Search
  • Backstage Catalogue
  • Backstage Tech Docs
  • Backstage Software Templates
  • Backstage Plug-ins

Backstage Search and Backstage Catalogue

For effective management and administration of software applications, the „Software Catalog“ feature of Backstage offers a centralized system for managing internal software within a company. It serves as a comprehensive listing of all developed applications, with each software component represented by its own entry. This allows for a quick overview of the company’s software availability. Each entry can be enriched with metadata to display dependencies between software components or to define the software type. The Software Catalog also facilitates the assignment of software to specific individuals, groups, or organizational units, making it easier to identify ownership and areas of responsibility. Moreover, each software component has various tabs that provide extensive information and management tools. Additionally, Backstage’s integrated „Backstage Search“ function enables searching the Backstage ecosystem for terms, components, people, and more, with the capability to extend search to the StackOverflow platform and integrate other search engines to customize search logic according to needs.

Screenshot of a search interface from an internal developer platform, possibly Spotify Backstage, showing a search bar with the query 'how to'. The results display a variety of programming and software development related questions such as 'How do I redirect to another webpage?', 'How to modify existing, unpushed commit messages?', 'How do I force "git pull" to overwrite local files?', and 'How do I revert a Git repository to a previous commit?'. Each question is tagged with relevant programming languages and tools such as JavaScript, query, git-commit, git-rewrite-history, version-control, git-pull, git-fetch, git-checkout, and git-revert, and includes the number of answers provided and the author's name. This indicates a knowledge sharing platform where developers can find answers to common coding and version control questions within their organization.
A screenshot of the Service Catalog interface from Spotify Backstage, showing an overview of services managed within an organization. The left pane categorizes services under 'Personal' with 32 owned and 2 starred, and under 'Spotify' with all services totaling 824. The main panel lists 'Owned (32)' services that the user currently manages, including services like 'podcast-api', 'artist-lookup', and 'searcher', each associated with a system, such as podcast or artist, and tagged with the tool or language used, like Python or Websites. The lifecycle stage for all listed services is 'Production', and their status varies with indications like 'Up and running', 'Increased failure rates', or 'Deployment failed'. Each service has a description, typically 'Graphql backend for Playlist', and actions for quick access like editing settings or viewing details. The interface also displays the current time in different cities, including NYC, UTC, LON, and STO.

Backstage Tech Docs

The „TechDocs“ feature is designed to aid in the management and administration of technical documentation, offering users a centralized platform to view and provide technical documentation for software components. Documentation can be accessed either through a tab on the respective software component’s page or via a button on the sidebar. Initially, developers write the documentation in the common Markdown format, which is then rendered for the website display using the „MkDocs“ Python module by Backstage. Users have the ability to structure the documentation with a table of contents, split the documentation across multiple Markdown files, and enhance TechDocs with additional functionalities through „TechDocs Addons“.

A screenshot of a documentation webpage titled 'Backend Golden Path' within an internal development platform, likely Spotify Backstage. The page is an introduction to a series on developing and operating server-side infrastructure. The main content explains that this tutorial is the first in the series and emphasizes the importance of setting up a local environment for backend development. It notes 'Operating System Variations', indicating that the tutorials are tailored for Apple machines running macOS, with alternative instructions for Linux or Windows users mentioned in callouts. The sidebar on the right includes tags related to the content such as #backend, #apollo, and #golden-path, and a section titled 'Stack Overflow' indicating no questions found for these tags. Other sections include 'Prerequisites' and 'Support'. The page header shows the component name 'backend-golden-path', the site owner 'pulp-fiction', the page owner 'fabric', and indicates the page was last updated a month ago. The top contributors are also acknowledged, symbolizing a collaborative documentation environment.

Backstage Software Templates

„Software Templates“ provide a smart way for creating new software or infrastructure components. Users are guided through the creation process with a pre-made template, where they can configure the component at each step by entering parameters, culminating in its creation with a simple button click. The output generated by the template is determined by its creator and can vary, including loading skeleton code, creating a new Git repository, or writing input parameters to a file. Additionally, there’s an option to register the newly created component in the Software Catalog. The process can be further customized to meet specific requirements by writing custom template actions, allowing organizations to establish standards and best practices for the creation of certain components.

A screenshot of a user interface from an internal development platform titled 'Create a New Component', showing a selection of available templates for software development. The 'Available Templates' section highlights 'Recommended' templates, including 'Documentation Template', 'React SSR Template', and 'Spring Boot gRPC Service'. Each template has a brief description, the owner (example@example.com), and associated tags like 'recommended', 'techdocs', 'mkdocs', 'website', 'react', 'service', 'java', and 'gRPC'. Below, in the 'Other Templates' section, more options are shown such as 'Create React App Template' and 'Pull Request Action Template'. The interface is designed for easy navigation, with categories and tags on the left for filtering the templates, and prominent 'Choose' buttons for template selection. A sidebar includes options for registering an existing component and seeking support, illustrating a comprehensive tool for managing and scaffolding software projects within the organization.

Integrations and Plugins

Backstage facilitates the integration of external platforms and cloud providers onto a unified platform, leveraging various services that use external providers for storing or publishing Backstage-specific content. For example, the Backstage Software Catalog can access external providers like GitHub to extract relevant information from YAML files, necessary for incorporating software components into the Software Catalog. Fundamentally, Backstage operates as a „single-page“ application, providing a technical framework in which various plugins, implementing desired functionalities, operate. These plugins, including „Core-Features“ provided by default installed plugins, can be extended with additional services via 3rd-party plugins available on the Marketplace. This Marketplace, accessible through the Backstage website, features plugins developed by Spotify, other organizations, or individuals, offering over 134 plugins across categories like „CI/CD,“ „Monitoring,“ „Infrastructure,“ or „Deployment.“ Plugins are typically installed using a package manager like npm, and by manually adding code to Backstage’s source code, integrating them into the existing tooling environment. Examples from the Marketplace include „Scaffolder Git Actions“ Hill or „Argo CD“. While plugins available in the Marketplace are generally free, Spotify also offers „Premium-Plugins“ as part of a paid subscription, developed by Spotify developers and supported officially by Spotify, such as the „Role-Based-Access-Control (RBAC)“ or „Soundcheck“ plugins.

A screenshot of the Backstage platform's plugin marketplace. The marketplace showcases a grid of plugin cards against a dark background, each card representing a different plugin with a distinct icon, title, and brief description. Examples include 'APIs with 3scale' by Red Hat for synchronizing 3scale content, 'Architecture Decision Records' by Phil Kuang for browsing project ADRs, and 'AI Assistant - RAG AI' marked as new, for querying language models with contextual information. Others include 'Allure Reports' by Deepak Bhardwaj for viewing reports, various analytics modules like Google Analytics by Spotify, Matomo Analytics by Red Hat, and Quantum Metric for tracking usage, along with monitoring tools such as 'Airbrake' by Simply Business. Additional plugins featured are 'Announcements' by procore-oss for writing and sharing announcements, 'API Docs' by SDA SE for API discovery, 'API Linter' by Zalando for compliance checking, and 'API SDK Generator' by Konfig for SDK generation. Each card has an 'Explore' button, facilitating easy access to the plugin details and installation options. The overall layout is user-friendly and informative, aiming to enhance the development environment with various integrations and tools.