Installing And Configuring Propagator: Part 2

So now that our set up is ready, we have the master server and two mirror servers. And before proceeding any further I Strongly suggest you to go through the README section in this link  https://github.com/KDE/propagator and if you feel like you can come back .

1.Foremost you need to have the following installed in you master server :

  • Python 3.5
  • RabbitMQ
  • Celery 3
  • GitPython : make sure first you install pip3 using sudo apt-get install python3-pip then you can pip3 install gitpython
  • Supervisord
  • Propagator uses the AsyncIO module : pip3 install asyncio
  • Other dependencies : pip3 install requests                           

NOTE : Also install pip3 and Gitpython in the two mirror servers as well.

2. Install propagator in the master VM for that first you need to clone it                                     git clone https://github.com/KDE/propagator.git and then configure the ~/propagator/config/ServerConfig.json  for that first you need to change the RepoRoot to the location you have stored the git repository. Export the variable GATOR_CONFIG_FILE in bash for that run the command  export GATOR_CONFIG_FILE=/path_to_propagator/config/ServerConfig.json          Then you can run the server with the command  supervisord -c ./supervisord.conf 

Screenshot from 2016-05-16 17-43-15

3.Setting up the AnongitRemote plugin to push to the two mirror servers :

  • First we generate a new SSH key in the master VM and make sure you store it somewhere else not ~/.ssh so that now we have two SSH keys on the server the old one is already in use so we keep it untouched now that we have a new SSH key we append the public key to the ~/.ssh/authorized_keys on the master server and configure the ~/propagator/config/ServerConfig.json  by setting the path of the private key in the field AnongitAPIKeyFile .And then we copy this public key to the two mirrors  with the command ssh-copy-id -i ~/path_to_new_sshkey/id_rsa.pub user@IP
  • Then we copy the agent/GatorSSHAgent from the master VM to the two mirror server, run this in the master VM                                                                                                scp /path_to_propagator/agent/GatorSSHAgent user@IP:/destination                         here user and IP are of the mirror VM and destination is the location you want to store it in mirror. Make sure you have installed pip3 and GitPython in the mirrors as well.
  • Now configure the ~/.ssh/authorized_keys file of the two mirrors. Note that the second ssh-rsa key is the newly copied key so we need to prefix the second key and it will look like  command=path/to/GatorSSHAgent ssh-rsaAcd ……..fddhg@ssh-key-comment

4. Lastly we need to edit the post-update hook of the git repository in the master server where we specify the location of mirrorctl in master server and the loacation of the git repo in mirror servers so add this command /path_to_propagator/bin/mirrorctl update user@IP:/location_of_git_repo name_of_repo.git                

DEBUGGING :

Please make sure you use Python 3 throughout as Propagator is written in Python3.5 if you get any errors during running it that means some modules you installed is not Python 3 compatible.All you need to do is to check the log files and run the modules in server separately eg : python3 CeleryWorkers.py  and in case you get errors debug accordingly. If you feel that the server is still not working delete /home/user/.propagator.sock and then try to run propagator again.

 

Setting Up Test Benches For Propagator: Part 1

1.We need to set three Virtual Machines that’s easy you can see the related tutorial on youtube.

Now the thing is we need to ssh from host(my physical machine) to the guest(master VM) and from the master VM to the two mirror VMs without using password. For that we need IP of the VMs. So the plan is to create two NICs, one NIC  is connected to NAT, and the second NIC is connected to vboxnet0 through Host-Only. First we go to File->Preferences setting in Virtual Box and then Network->Host Only Networks add a new network. Then we need to make sure that all the VMs are connected to two Ethernet interfaces.

If you are using any server distribution on your VM you need to add the new network in the file /etc/network/interfaces.

That’s it now you have the IP to ssh.

Screenshot from 2016-05-15 00-21-10.png

2.Install openssh-client on the host machine and openssh-server on the master VM and two mirror VM’s  .

Generate two RSA-4096 keypairs – one on the host machine, one on the master server using the command  ssh-keygen -t rsa -b 4096

Set up the master server with the public key of your host machine, and set up the mirrors with the public key of the master machine so we can use ssh-copy-id -i ~/.ssh/id_rsa.pub user@IP and to ssh without password we need to change the permission of .ssh/authorized_keys and add the private key identities to the authentication agent, use ssh-add

Screenshot from 2016-05-15 00-27-06.png

3.Set up git server on all the three VMs, first sudo apt-get install git-core on all the three servers then create bare repository  first cd into the git repository then git –bare init and set up post-update hook. For that you need to create a post-update file in which we write the bash script to push the changes to the mirror servers. The contents of the post-update file on the master git server .

Screenshot from 2016-05-15 00-23-00

As we see here we have used git push –mirror url  command which pushes the committed changes to the specified mirror url from the master server to the two mirror servers.

Testing :

We can now clone the repository from the master to our host machine and can push the commits and we have the changes in the mirror servers as well.

You can also check using a different branch to commit changes and then check that the changes are reflected on both the mirror git servers.

Screenshot from 2016-05-16 03-44-04