A Kubernetes operator that monitors the health of HTTP endpoints by periodically sending requests and tracking their responses.

This entire repo is the result of me going "Hey, I wonder if I can vibe-code a Kubernetes operator in a weekend?"

Me over the weekend:

Buddy-coded with Cursor Pro, 307 premium model requests.

The Heartbeats Operator provides a custom resource Heartbeat that allows you to monitor the health of HTTP endpoints in your Kubernetes cluster. It periodically checks the endpoints and updates their status based on HTTP response codes with comprehensive error handling and reporting.

• Robust Health Monitoring: Monitor HTTP endpoints with configurable intervals and retry logic
• Flexible Status Code Ranges: Define custom status code ranges for healthy/unhealthy states
• Secure Configuration: Store endpoint URLs securely in Kubernetes secrets
• Real-time Status Updates: Detailed health information with comprehensive error messages
• Configurable Timeouts: Adjustable timeout and retry settings for different network conditions
• Comprehensive Error Handling: Detailed error messages for troubleshooting
• Report Endpoints: Reporting to healthy/unhealthy endpoints for external monitoring systems
• Controller-runtime metrics: HTTPS /metrics on port 8443

• Kubernetes cluster
• kubectl configured to access your cluster

make install
make deploy IMG=ghcr.io/siutsin/heartbeats:latest

make install applies the CRD. make deploy builds config/default with kustomize.

1. Create a secret containing your endpoint URLs:

   apiVersion: v1
   kind: Secret
   metadata:
     name: heartbeat-endpoints
   type: Opaque
   stringData:
     targetEndpoint: "https://api.example.com/health"
     healthyEndpoint: "https://httpbin.org/status/200"
     unhealthyEndpoint: "https://httpbin.org/status/500"

2. Create a Heartbeat resource:

   apiVersion: monitoring.siutsin.com/v1alpha1
   kind: Heartbeat
   metadata:
     name: api-health
   spec:
     endpointsSecret:
       name: heartbeat-endpoints
       targetEndpointKey: targetEndpoint
       healthyEndpointKey: healthyEndpoint
       unhealthyEndpointKey: unhealthyEndpoint
       healthyEndpointMethod: GET
       unhealthyEndpointMethod: GET
     expectedStatusCodeRanges:
       - min: 200
         max: 299
     interval: 30s

apiVersion: monitoring.siutsin.com/v1alpha1
kind: Heartbeat
metadata:
  name: api-health-with-reporting
spec:
  endpointsSecret:
    name: heartbeat-endpoints
    targetEndpointKey: targetEndpoint
    healthyEndpointKey: healthyEndpoint
    unhealthyEndpointKey: unhealthyEndpoint
    healthyEndpointMethod: GET
    unhealthyEndpointMethod: GET
  expectedStatusCodeRanges:
    - min: 200
      max: 299
    - min: 404
      max: 404  # Accept 404 as healthy for certain endpoints
  interval: 30s

The Heartbeat resource's status includes:

• healthy: Boolean indicating if the endpoint is healthy
• lastStatus: Last HTTP status code received
• message: Human-readable status message with detailed error information
• lastChecked: Timestamp of the last health check
• reportStatus: Status of reporting to external endpoints ("Success" or "Failure")

The operator provides detailed error messages for troubleshooting:

• missing required key: A required endpoint key is missing from the secret
• endpoint is not specified: An endpoint URL is empty or not provided
• failed to check endpoint health: Network or HTTP errors during health checks
• endpoint timed out: The endpoint took too long to respond
• invalid status code range: Status code range has invalid min/max values
• status code is not within expected ranges: Endpoint returned unexpected status code

• Go 1.24+
• Docker
• Kind (for local testing)
• kubectl

# Run unit tests
make test

# Run unit tests with race detection (for CI)
make test-ci

# Run e2e tests locally
make test-e2e LOCAL=true

# Run e2e tests with race detection (for CI)
make test-e2e-ci LOCAL=true

# Build the operator
make build

# Build Docker image
make docker-build

# Format code
make fmt

# Run linter
make lint

# Run linter with fixes
make lint-fix

# Check markdown files
make lint-markdown

The operator logs JSON to stdout via slog.

• INFO: General operational information, health check results
• ERROR: Error conditions, failed health checks, configuration issues
• DEBUG/V(1): Detailed debugging information, HTTP request details

The operator exposes controller-runtime metrics at /metrics on port 8443.

The metrics endpoint is secured and requires authentication. Access is controlled via RBAC:

# Get service account token
TOKEN=$(kubectl create token heartbeats-operator-controller-manager -n heartbeats-operator-system)

# Access metrics
curl -k -H "Authorization: Bearer $TOKEN" \
  https://heartbeats-operator-controller-manager-metrics-service.heartbeats-operator-system.svc.cluster.local:8443/metrics

1. Endpoint Timeout

   • Check if the endpoint is accessible from the cluster
   • Verify network policies allow the connection
   • Consider increasing the timeout duration
   • Check logs for detailed timeout information

2. Invalid Status Code

   • Ensure the endpoint returns expected status codes
   • Verify the status code ranges in the Heartbeat spec
   • Check the status message for specific error details

3. Secret Not Found

   • Confirm the secret exists in the correct namespace
   • Verify the secret keys match the Heartbeat configuration
   • Check for "missing required key" error messages

4. Report Endpoint Failures

   • Verify the report endpoints are accessible
   • Check the reportStatus field in the Heartbeat status
   • Review logs for report endpoint errors

# Check Heartbeat status
kubectl get heartbeat <name> -o yaml

# View operator logs
kubectl logs -n heartbeats-operator-system deployment/heartbeats-operator-controller-manager

# Check metrics
kubectl port-forward -n heartbeats-operator-system svc/heartbeats-operator-controller-manager-metrics-service 8443:8443

Contributions are welcome! Please see our Contributing Guide for details.

• Follow the existing code style and patterns
• Add comprehensive unit tests for new features
• Include e2e tests for integration scenarios
• Use structured logging with appropriate log levels
• Follow error handling patterns established in the codebase

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.