Skip to main content

Provisioning new services

This section is under construction

We're building documentation for this section.

When a developer requests the creation of a new service via nullplatform, the provisioning process is initiated. This also applies when updating or deleting a service.

nullplatform facilitates the dynamic provisioning, updating, and management of services through an orchestrated interaction between developers, nullplatform services, and various provisioning mechanisms. This process is designed to ensure seamless integration and operation within your existing infrastructure and workflows.

Below is a diagram that highlights the service creation process, which is provisioned using Github Actions and OpenTofu.

Overview of the Provisioning Process​

Provisioning Workflow

Initial Request by Developer​

A developer initiates the service lifecycle by invoking a specific action on the nullplatform. These actions can range from creating a new service, updating an existing service, to deleting a service, or even any custom action your organization defined (in this example: retrieving credentials). The available actions are:

  • Create MySQL: Initiates the creation of a MySQL service.
  • Update MySQL: Applies updates to an existing MySQL service.
  • Delete MySQL: Removes a MySQL service from the environment.
  • Get Credentials: Generate temporal credentials for the MySQL service.

In this particular case Create MySQL action was invoked.

Orchestration and Triggering Events​

Once an action is invoked, the nullplatform’s orchestrator takes over to manage the process flow. The orchestrator is a crucial component that manages:

  • Calling appropriate services based on the developer’s request.
  • Triggering further actions necessary to fulfill the request through various runners and Infrastructure as Code (IaC) tools.

Runners and IaC Tools​

The actual provisioning tasks are executed by runners and IaC tools, which are triggered by nullplatform but managed directly by the client, ensuring flexibility and compliance with security standards. The primary tools and mechanisms include:

  • Webhook: Custom webhooks can be utilized to trigger provisioning actions, offering a versatile method for integrating with existing systems.
  • GitLab Pipeline: Automated workflows in GitLab trigger necessary provisioning actions based on the defined pipeline configurations.
  • GitHub Actions: Similar to GitLab, GitHub Actions automate the workflow processes, facilitating the execution of provisioning tasks directly from GitHub repositories.
  • IaC tools: You can use any IaC tools to provision or manage resources directly from the code, aligning infrastructure management with application development.

So, invoking runners like Webhooks, GitLab Pipeline, and GitHub Actions are managed directly by nullplatform, providing a controlled and secure environment for service management, then provisioning tasks are executed by IaC tools like Pulumi, OpenTofu, and other tools are managed by the client, allowing them to maintain control over their infrastructure and adhere to their operational protocols.

Payload example​

Below is an example of a notification payload, formatted consistently whether it is triggered by a webhook, a GitHub Action workflow dispatch, or a GitLab pipeline:

{
  "id": "a04c2970-47b7-4ae8-a58b-a7e8b2781b10",
  "source": "service",
  "created_at": "1714683395605",
  "notification": {
    "action": "service:action:create",
    "id": "70afee17-cef2-49e0-8931-ae6de3df3b96",
    "name": "create-users-queue",
    "slug": "create-users-queue",
    "messages": [],
    "status": "pending",
    "created_at": "2024-05-02T20:56:35.389Z",
    "updated_at": "2024-05-02T20:56:35.389Z",
    "parameters": {
      "fifo": false,
      "dead_letter": false
    },
    "results": {},
    "type": "create",
    "specification": {
      "id": "284c262a-dcf4-4af8-9dca-7c310c57d42b",
      "slug": "create-sqs-queue"
    },
    "service": {
      "id": "363a5893-ed84-4eba-a4c1-a80098b9f74e",
      "slug": "users-queue",
      "attributes": {},
      "specification": {
        "id": "529d8786-4af4-4625-87de-664ad7c9ef5f",
        "slug": "sqs-queue"
      },
      "dimensions": {
        "environment": "production",
        "country": "us"
      }
    },
    "link": null,
    "tags": {
      "organization_id": "17126128",
      "organization": "my-organization",
      "account_id": "1264712817",
      "account": "my-account",
      "namespace_id": "812712367",
      "namespace": "my-namespace",
      "application_id": "219123817",
      "application": "my-application",
      "scope_id": "1761267173",
      "scope": "my-scope",
    }
    "entity_nrn": "organization=1:account=1:namespace=1:application=1"
  }
}

GitHub Actions​

GitHub Actions can be used for provisioning services on nullplatform. Below are the steps to set up and use GitHub Actions for this purpose:

  1. Create a Repository

    • Start by creating a new repository on GitHub to host your service provisioning workflows.

  2. Create a GitHub Action Workflow

    • Set up a GitHub Action workflow that can be triggered manually. This workflow will handle the provisioning tasks based on the inputs it receives. Here’s an example configuration for a service provisioning workflow:
    name: Service Provisioning
    on:
      workflow_dispatch:
        inputs:
          context:
            type: string
            description: 'nullplatform service lifecycle webhook context'
            required: true
    permissions:
      id-token: write
      contents: read
      packages: read
    jobs:
      provisioning:
        runs-on: ubuntu-latest
        env:
          CONTEXT: ${{ inputs.context }}
        steps:
          - name: Provisioning
          - run: echo "Insert your provisioning code here"

  3. Integration with nullplatform

    • Create a notification channel to integrate this workflow with your service provisioning process. 
      POST /notification/channel

      {
         "nrn": "organization=1:account=2",
         "source": [
            "service"
         ],
         "type": "github",
         "configuration": {
            "installation_id": "my-installation-id",
            "account": "my-github-org",
            "repository": "services-terraform",
            "reference": "main",
            "workflow_id": "provisioning.yml"
         }
      }
    • After creating the channel on nullplatform, this workflow will be triggered automatically each time a service or link is created.

  4. Workflow Execution

    • The payload example provided earlier will be available as the context input in your workflow.

  5. Security and Cost Management

    • It is strongly recommended to use your own runners for enhanced security and cost management.

Gitlab Pipelines​

Gitlab Pipelines can also be used for provisioning services on nullplatform. Below are the steps to set up and use Gitlab Pipelines for this purpose:

  1. Create a Repository

    • Start by creating a new repository on Gitlab to host your service provisioning workflows.

  2. Create a Gitlab Pipeline

    • Set up a Gitlab Pipeline. This pipeline will handle the provisioning tasks based on the inputs it receives. Here’s an example configuration for a service provisioning workflow:
    default:
      tags:
        - $RUNNER_TAG
    provisioning:
      stage: provisioning
      script:
        - echo $CONTEXT
        - echo "Insert your provisioning code here"

  3. Integration with nullplatform

    • Create a notification channel to integrate this pipeline with your service provisioning process. 
      POST /notification/channel

      {
         "nrn": "organization=1:account=2",
         "source": [
            "service"
         ],
         "type": "gitlab",
         "configuration": {
            "reference": "main",
            "project_id": "1"
         }
      }
    • After creating the channel on nullplatform, this pipeline will be triggered automatically each time a service or link is created.

  4. Workflow Execution

    • The payload example provided earlier will be available as the CONTEXT environment variable in your pipeline.

  5. Security and Cost Management

    • It is strongly recommended to use your own runners for enhanced security and cost management.

Azure DevOps Pipelines​

Azure DevOps Pipelines can also be used for provisioning services on nullplatform. Below are the steps to set up and use Azure DevOps Pipelines for this purpose:

  1. Create a Repository

    • Start by creating a new repository on Azure DevOps to host your service provisioning workflows.

  2. Create a Pipeline

    • Set up an Azure DevOps Pipeline. This pipeline will handle the provisioning tasks based on the inputs it receives. Here’s an example configuration for a service provisioning workflow:
    trigger:
    - main

    pool:
      vmImage: ubuntu-latest

    steps:
    - script: |
        echo $(CONTEXT)
        echo "Insert your provisioning code here"
      displayName: 'Run Provisioning Scripts'
    • Ensure you set up a variable CONTEXT in the pipeline configuration with the option "Let users override this value when running this pipeline" enabled.

  3. Integration with nullplatform

    • Create a notification channel to integrate this pipeline with your service provisioning process. 
      POST /notification/channel

      {
         "nrn": "organization=1:account=2",
         "source": [
            "service"
         ],
         "type": "azure",
         "configuration": {
           "organization": "my-org",
           "project": "my-devops-project",
           "pipeline_id": 1,
           "reference": "main"
         }
      }
    • After creating the channel on nullplatform, this pipeline will be triggered automatically each time a service or link is created.

  4. Workflow Execution

    • The payload example provided earlier will be available as the CONTEXT environment variable in your pipeline.

  5. Security and Cost Management

    • It is strongly recommended to use your own runners for enhanced security and cost management.

Webhook notification​

Webhooks can be used for provisioning services on nullplatform. Below are the steps to set up and use webhooks for this purpose:

  1. Create a web server

    • Set up a web server with a POST endpoint to receive webhooks from nullplatform. The endpoint should be able to parse the payload and trigger the provisioning tasks based on the inputs it receives.
    • Respond to nullplatform with an HTTP 2XX status code (e.g., 200 OK or 201 Created) to indicate successful receipt of the webhook. Processing should be done asynchronously to avoid timeouts because it is a time-consuming task.

  2. Integration with nullplatform

    • Create a Http notification channel to integrate this workflow with your service provisioning process. 
      POST /notification/channel

      {
         "nrn": "organization=1:account=2",
         "source": [
            "service"
         ],
         "type": "http",
         "configuration": {
            "url": "https://yourdomain.com/url-you-configured",
            "headers": {
               "Authorization": "Bearer the-token"
            }
         }
      }
    • After creating the channel on nullplatform, this endpoint will be requested automatically each time a service or link is created.

  3. Webhook Invocation

    • The payload example provided earlier will be available in the body of the request.
    • Ensure task execution handles edge cases and failures gracefully.

  4. Notify nullplatform the result of the webhook execution

    • Use the nullplatform API to update the webhook execution results of the service action or link action, providing detailed outcomes and messages for the developer.
    • To interact with the API you will need an API Key with the rol 650969136 (nullplatform:services-runner-terraform).

Additional Configuration for OpenTofu/Terraform IaC Tools​

If you are using OpenTofu/Terraform as infrastructure as code tool you will need the following additional setups:

  • Setup Locking Mechanism

    • If running on AWS, utilize DynamoDB to lock the state during operations to prevent conflicts.

  • Use Remote State Provider

    • Use Amazon S3 as a remote state provider if your infrastructure is on AWS to ensure all state files are securely managed and accessible.

  • Unique State Management

    • Maintain a unique state for each service and link. Utilize the service/link ID and slug to generate unique keys, facilitating distinct management of each entity's configuration.

By following these guidelines, you can leverage GitHub Actions to automate and secure your service provisioning processes on nullplatform, ensuring a robust and reliable infrastructure management environment.

Key Observations:​

  • Tool Agnosticism: nullplatform is not opinionated about the provisioning tools used; it can integrate with any Infrastructure as Code (IaC) tool, various SDKs, or even bespoke systems specific to an organization.
  • Orchestration Management: nullplatform solely manages the orchestration process, storing only the necessary metadata. The actual provisioning occurs within the client's infrastructure, adhering to their security standards and definitions.
  • Uniform Handling of Actions: All actions—whether creation, editing, deletion, or even custom-defined actions—are treated equally and processed by our provisioning system.
  • Event Triggering: Our provisioning system triggers events with the necessary information for the organization to carry out the work. These events can be triggered via GitHub Actions, GitLab Pipelines, or configured to be received through a webhook. Regardless of the mechanism, the information sent remains consistent.