Configuring Sidekiq

This section discusses how to configure an external Sidekiq instance.

Sidekiq requires connection to the Redis, PostgreSQL and Gitaly instance. To configure the Sidekiq node:

  1. SSH into the Sidekiq server.

  2. Download/install the Omnibus GitLab package you want using steps 1 and 2 from the GitLab downloads page. Do not complete any other steps on the download page.

  3. Open /etc/gitlab/gitlab.rb with your editor.

  4. Generate the Sidekiq configuration:

    sidekiq['listen_address'] = "10.10.1.48"
    
    ## Optional: Enable extra Sidekiq processes
    sidekiq_cluster['enable'] = true
    sidekiq_cluster['enable'] = true
      "elastic_indexer"
    ]
  5. Setup Sidekiq's connection to Redis:

    ## Must be the same in every sentinel node
    redis['master_name'] = 'gitlab-redis'
    
    ## The same password for Redis authentication you set up for the master node.
    redis['master_password'] = 'YOUR_PASSOWORD'
    
    ## A list of sentinels with `host` and `port`
    gitlab_rails['redis_sentinels'] = [
        {'host' => '10.10.1.34', 'port' => 26379},
        {'host' => '10.10.1.35', 'port' => 26379},
        {'host' => '10.10.1.36', 'port' => 26379},
      ]
  6. Setup Sidekiq's connection to Gitaly:

    git_data_dirs({
      'default' => { 'gitaly_address' => 'tcp://gitaly:8075' },
    })
    gitlab_rails['gitaly_token'] = 'YOUR_TOKEN'
  7. Setup Sidekiq's connection to PostgreSQL:

    gitlab_rails['db_host'] = '10.10.1.30'
    gitlab_rails['db_password'] = 'YOUR_PASSOWORD'
    gitlab_rails['db_port'] = '5432'
    gitlab_rails['db_adapter'] = 'postgresql'
    gitlab_rails['db_encoding'] = 'unicode'
    gitlab_rails['auto_migrate'] = false

    Remember to add the Sidekiq nodes to the PostgreSQL whitelist:

    postgresql['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.10.1.30/32 10.10.1.31/32 10.10.1.32/32 10.10.1.33/32 10.10.1.38/32)
  8. Disable other services:

    nginx['enable'] = false
    grafana['enable'] = false
    prometheus['enable'] = false
    gitlab_rails['auto_migrate'] = false
    alertmanager['enable'] = false
    gitaly['enable'] = false
    gitlab_monitor['enable'] = false
    gitlab_workhorse['enable'] = false
    nginx['enable'] = false
    postgres_exporter['enable'] = false
    postgresql['enable'] = false
    redis['enable'] = false
    redis_exporter['enable'] = false
    puma['enable'] = false
    gitlab_exporter['enable'] = false
  9. Run gitlab-ctl reconfigure.

NOTE: Note: You will need to restart the Sidekiq nodes after an update has occurred and database migrations performed.

Example configuration

Here's what the ending /etc/gitlab/gitlab.rb would look like:

########################################
#####        Services Disabled       ###
########################################

nginx['enable'] = false
grafana['enable'] = false
prometheus['enable'] = false
gitlab_rails['auto_migrate'] = false
alertmanager['enable'] = false
gitaly['enable'] = false
gitlab_monitor['enable'] = false
gitlab_workhorse['enable'] = false
nginx['enable'] = false
postgres_exporter['enable'] = false
postgresql['enable'] = false
redis['enable'] = false
redis_exporter['enable'] = false
puma['enable'] = false
gitlab_exporter['enable'] = false

########################################
####              Redis              ###
########################################

## Must be the same in every sentinel node
redis['master_name'] = 'gitlab-redis'

## The same password for Redis authentication you set up for the master node.
redis['master_password'] = 'YOUR_PASSOWORD'

## A list of sentinels with `host` and `port`
gitlab_rails['redis_sentinels'] = [
    {'host' => '10.10.1.34', 'port' => 26379},
    {'host' => '10.10.1.35', 'port' => 26379},
    {'host' => '10.10.1.36', 'port' => 26379},
  ]

#######################################
###              Gitaly             ###
#######################################

git_data_dirs({
  'default' => { 'gitaly_address' => 'tcp://gitaly:8075' },
})
gitlab_rails['gitaly_token'] = 'YOUR_TOKEN'

#######################################
###            Postgres             ###
#######################################
gitlab_rails['db_host'] = '10.10.1.30'
gitlab_rails['db_password'] = 'YOUR_PASSOWORD'
gitlab_rails['db_port'] = '5432'
gitlab_rails['db_adapter'] = 'postgresql'
gitlab_rails['db_encoding'] = 'unicode'
gitlab_rails['auto_migrate'] = false

#######################################
###      Sidekiq configuration      ###
#######################################
sidekiq['listen_address'] = "10.10.1.48"

#######################################
###     Monitoring configuration    ###
#######################################
consul['enable'] = true
consul['monitoring_service_discovery'] =  true

consul['configuration'] = {
  bind_addr: '10.10.1.48',
  retry_join: %w(10.10.1.34 10.10.1.35 10.10.1.36)
}

# Set the network addresses that the exporters will listen on
node_exporter['listen_address'] = '10.10.1.48:9100'

# Rails Status for prometheus
gitlab_rails['monitoring_whitelist'] = ['10.10.1.42', '127.0.0.1']

Further reading

Related Sidekiq configuration:

  1. Extra Sidekiq processes
  2. Using the GitLab-Sidekiq chart