Running SphinxSearch in Podman container

This post documents the setup of SphinxSearch indexing the Postgresql database. The whole installation is hosted on Fedora using Podman containers in a pod.

mkdir -p ~/podman/volumes/sphinxsearch/index
sudo podman pull macbre/sphinxsearch

Prepare the config file, and put it under ~/podman/volumes/sphinxsearch directory eg.

source jrnl
{
    type            = pgsql

    sql_host        = 127.0.0.1
    sql_user        = $DBUSER
    sql_pass        = $DBPASS
    sql_db          = $DBNAME_TO_BE_INDEXED
    sql_port        = 5432  

    sql_query       = \
        SELECT id, body \
        FROM jrnl

}


index jrnl
{
    source          = jrnl
    path            = /opt/sphinx/index/jrnl
}

indexer
{
    mem_limit       = 128M
}


searchd
{
    listen          = 9312
    listen          = 9306:mysql41
    log             = /opt/sphinx/index/searchd.log
    query_log       = /opt/sphinx/index/query.log
    read_timeout    = 5
    max_children    = 30
    pid_file        = /opt/sphinx/index/searchd.pid
    seamless_rotate = 1
    preopen_indexes = 1
    unlink_old      = 1
    workers         = threads # for RT to work
    binlog_path     = /opt/sphinx/index
}

Run the first indexer job through the container to build the index:

sudo podman run --pod $PODNAME -v /home/$USER/podman/volumes/sphinxsearch/index/:/opt/sphinx/index:z -v /home/$USER/podman/volumes/sphinxsearch/sphinx.conf:/opt/sphinx/conf/sphinx.conf:z --rm macbre/sphinxsearch indexer --all --config /opt/sphinx/conf/sphinx.conf

To make it possible for SphinxSearch to connect to PostgreSql you have to add them to the one pod. If you haven't already created dedicated pod for Postgres and SphinxSearch publishing its ports then issue the commands:

sudo podman pod create --name $PODNAME -p 5432:5432 -p 9306:9306
sudo podman run --name postgres12 --pod $PODNAME -e POSTGRES_PASSWORD=docker -d -v $HOME/podman/volumes/postgres12:/var/lib/postgresql/data:z postgres:12

The commands are run via sudo, as they will be later added to systemctl to make it system wide service.

Now run the container and attach it to the pod. 

$ sudo podman run --name sphinxsearch --pod $PODNAME -d -v /home/$USER/podman/volumes/sphinxsearch/index/:/opt/sphinx/index:z -v /home/$USER/podman/volumes/sphinxsearch/sphinx.conf:/opt/sphinx/conf/sphinx.conf:z macbre/sphinxsearch
# Check the startup logs
$ sudo logs sphinxsearch

Test the index:

$ sudo dnf install mariadb
$ mysql -h127.0.0.1 -P9306
 > select * from jrnl where match('linux');

Systemctl

Configure systemd service to start container during the system startup

 $ sudo setsebool -P container_manage_cgroup on
 $ sudo podman generate systemd --name $PODNAME
 # create services
 $ sudo vim /etc/systemd/system/pod-$PODNAME.service
 $ sudo vim /etc/systemd/system/container-postgres12.service
 $ sudo vim /etc/systemd/system/container-sphinxsearch.service

 $ sudo systemctl enable pod-$PODNAME.service
 $ sudo systemctl enable container-postgres12.service
 $ sudo systemctl enable container-sphinxsearch.service

If you already have the pod with Postgresql container and with published port 9306 it is enough to add container-sphinxsearch.service and update the pod-$PODNAME.service (Requires and Before) then do systemctl daemon-reload.

Check if everything works fine:

$ systemctl stop pod-$PODNAME
$ systemctl start pod-$PODNAME

and finally reboot the system to check if it starts properly.

Update index

To update index periodically with new data add to the crontab:

* */2 * * * sudo podman exec -it sphinxsearch indexer --rotate --all --config /opt/sphinx/conf/sphinx.conf > /tmp/sphinxsearch-indexer.log 2>&1

References