Skip to main content

Integrate

Back to SDLC Intro

The Integrate phase involves combining individual software modules into a unified group. In modern , this is often continuous (), but conceptually it represents the assembly of the system. The goal is simple: every change should be integrated, built, and validated automatically - so that the team always has a working, deployable product.

CI Pipeline Design​

A well-designed CI pipeline is the backbone of a healthy engineering organisation. It should provide fast, reliable feedback on every change.

Pipeline Stages​

A typical CI pipeline follows this sequence:

  1. Checkout: Pull the latest code and dependencies.
  2. Install: Resolve and install dependencies from lock files.
  3. Lint and Format: Catch style violations and formatting issues instantly.
  4. Build: Compile the application and verify it produces valid .
  5. Unit Tests: Run the fast, isolated test suite.
  6. Integration Tests: Verify cross-module and cross-service contracts.
  7. Security Scan: Check dependencies and code for known vulnerabilities.
  8. Artifact Publish: Store versioned build artifacts for downstream deployment.
Pipeline as Code

Define your CI pipeline in version-controlled configuration files (e.g. .github/workflows/*.yml, .gitlab-ci.yml, Jenkinsfile) rather than configuring it through a UI. This ensures pipelines are reviewable, auditable, and reproducible. is a non-negotiable best practice.

CI Best Practices​

  • Fast Feedback: The pipeline should complete in under 10 minutes for the core feedback loop (lint + build + unit tests). Longer-running tests (integration, E2E) can run in parallel or as a secondary stage.
  • Fail Fast: Order pipeline stages so the cheapest and fastest checks run first. If linting fails, there is no point running a 30-minute E2E suite.
  • Caching: Cache dependency installations (node_modules, pip cache, Maven repository) between runs to dramatically reduce build times.
  • Parallelisation: Run independent test suites in parallel. Most CI platforms support matrix builds and parallel job execution.
  • Deterministic Builds: Use lock files (package-lock.json, yarn.lock, Pipfile.lock) and pinned dependency versions to ensure the same input always produces the same output.

Dependency Management​

Modern applications depend on hundreds of third-party packages. Managing these dependencies is a critical integration activity.

  • Lock Files: Always commit lock files to version control. They ensure every developer and CI run uses exactly the same dependency versions.
  • Vulnerability Scanning: Automate dependency vulnerability checks using tools like Snyk, Dependabot, or Renovate. These tools can automatically create PRs to update vulnerable packages.
  • Update Policies: Define a cadence for dependency updates. Weekly automated PRs for patch/minor versions, manual review for major versions. Never let dependencies go unpatched for months.
  • License Compliance: Track the licences of all third-party dependencies. Some licences (, ) have implications for commercial software that must be reviewed by legal.

Monorepo vs Polyrepo​

How you organise your source code has significant implications for integration:

  • : All services and packages in a single repository. Enables atomic cross-service changes, shared tooling, and simplified dependency management. Requires investment in build tooling (Nx, Turborepo, Bazel) to keep builds fast.
  • : Each service in its own repository. Clearer ownership boundaries and simpler per-repo CI. Cross-service changes require coordinated PRs and versioned contracts.
By Company Stage
  • Startup: Use a single repository with a simple CI pipeline (GitHub Actions or similar). One pipeline file, one build, one deploy. Keep it fast and simple. Minimal gates - lint, build, test, deploy. The overhead of managing multiple repos is not worth it at this stage.
  • Growth Stage: Introduce multi-stage pipelines with separate jobs for different test types. Set up an (GitHub Packages, JFrog Artifactory, AWS ECR). Start generating . Consider a monorepo with Nx or Turborepo if teams are sharing significant code.
  • Established: Invest in enterprise CI platforms (Jenkins at scale, GitLab Ultimate, CircleCI with self-hosted runners). Cross-repo integration testing for architectures. Compliance scanning (licence checks, security gates) as mandatory pipeline stages. Dedicated platform/DevEx team to maintain CI infrastructure. generation and attestation for supply chain security.

Common Pitfalls​

Anti-patterns to Avoid
  • Slow Pipelines: If CI takes 30+ minutes, developers stop waiting for it and merge without feedback. Invest in caching, parallelisation, and splitting fast/slow test stages.
  • Ignoring Flaky Tests: A test suite with 5% flake rate means teams learn to ignore red builds. This erodes trust and lets real bugs through. Track and fix flaky tests aggressively.
  • No Artifact Versioning: Building from source on every deploy (rather than promoting a tested artifact) introduces risk and wastes time. Build once, deploy the same artifact everywhere.
  • "It Merged, It's Done": Integration is not just about merging code. It includes validating that the merged result builds, passes tests, and produces a deployable artifact. A green merge button is not a quality gate on its own.

Integration Key Deliverables​

  • Build Artifacts (, , , bundles)
  • Integrated Codebase (green main branch)
  • Software Bill of Materials ()
  • CI Pipeline Configuration (as code)
How AI Can Help: Integration

AI streamlines specific pain points in integration:

  • Conflict Resolution: AI can analyze code logic to suggest optimal merge strategies and resolve conflicts (e.g., previously mentioned IDEs in the Code section or features in GitLab or GitHub).
  • Smart Builds: Tools like Incredibuild use AI to distribute build tasks efficiently, significantly reducing build times.
  • SBOM Management: AI helps generate and analyze for security risks, with tools like Sonatype Nexus and JFrog Artifactory providing real-time intelligence on open-source components.
  • Delivery and flow metrics: Tools like LinearB integrate with Git and project management to measure delivery flow and -style metrics during integration and release.