Set Up COPO Project on Linux

Note

If you are using a different OS other than Ubuntu, you can skip this step. See COPO project guidelines for Windows users if Windows is your preferred OS.

Install Python 

sudo apt -y install software-properties-common
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt -y install python3.8

Install Python Development Tools 

sudo apt -y install python3.8-distutils
sudo apt -y install build-essential libssl-dev libffi-dev
sudo apt -y install libxml2-dev libxslt1-dev python3-django
sudo apt update && sudo apt install -y default-jre rsync git nano libxml2-dev python build-essential make gcc python3-dev locales python3-pip ruby-dev rubygems poppler-utils

Install Required Packages

Install required packages and enable the databases to start on boot 
sudo apt -y install supervisor
sudo gem install sass
sudo apt -y install redis mongodb postgresql
sudo systemctl enable mongodb
sudo systemctl enable postgresql
sudo systemctl enable redis
Install MongoDB on your local machine 
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
echo "deb http://repo.mongodb.org/apt/ubuntu "$(lsb_release -sc)"/mongodb-org/3.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.0.list
sudo apt update
sudo apt -y install mongodb-org

Install Integrated Development Environment (IDE)

Any IDE can be used to work on the project. However, we recommend using one of the following IDEs:

PyCharm Professional

A cross-platform IDE that provides consistent experience on the Windows, macOS, and Linux operating systems.

sudo snap install pycharm-professional --classic

OR

Visit PyCharm website to download an appropriate version for your OS or download it from the Ubuntu Software Center.


Visual Studio Code

VSCode is a lightweight but powerful source code editor which runs on your desktop and is available on Windows, macOS and Linux.

It comes with built-in support for JavaScript (JS), TypeScript and Node.js and has a rich ecosystem of extensions for other languages (such as C++, C#, Java, Python, PHP, Go) and runtimes (such as .NET and Unity).

Visit VSCode website to download an appropriate version for your OS or download it from the Ubuntu Software Center

Configure IDE

PyCharm

Note

If using VSCode IDE, you can skip this step.

Warning

If you encounter the error, Error: Please enable Django support for the project, navigate to File->Settings->Languages & Frameworks->Django->Enable Django to enable Django support for the project.

If using PyCharm IDE, add a new configuration by following the steps below:

  1. Navigate to Add New Configuration

  2. Select Django server

  3. Click Save

Visit Run/debug configurations to learn how to create a configuration in PyCharm.

VSCode

If using VSCode IDE, add a new configuration by following the steps below:

  1. Navigate to Run -> Add Configuration

  2. Edit the launch.json file that is created with the following file contents:

VSCode launch.json configuration file contents 
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Django",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/manage.py",    
            "env" : {
                "PYTHONPATH1" : "${workspaceFolder}/lib:${PYTHONPATH}"
            },      
            "args": [
                "runserver",
                "0.0.0.0:8000"
            ],
            "django": true,
            "justMyCode": false
        }, 
        {
            "name": "Python: Django Shell",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/manage.py",    
            "env" : {
                "PYTHONPATH1" : "${workspaceFolder}/lib:${PYTHONPATH}"
            },      
            "args": [
                "shell_plus",
                "--ptpython"
            ],
            "django": true,
            "justMyCode": false
        },     
        {
            "name": "Python: Django makemigration",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/manage.py",    
            "env" : {
                "PYTHONPATH" : "${workspaceFolder}/lib:${PYTHONPATH}"
            },      
            "args": [
                "makemigration"
            ],
            "django": true,
            "justMyCode": false
        },                
        {
            "name": "Python: Celery Workers",
            "type": "python",
            "env" : {
                "GEVENT_SUPPORT": "true"
            },
            "request": "launch",
            "module": "celery",
            "console": "integratedTerminal",
            "args": ["-A", "src", "worker", "-l", "debug", "-P", "gevent", "--concurrency" , "10", "-Q", "celery",  "-E", "--hostname=celery@%h"],
            "justMyCode": true
        },
        {
            "name": "Python: Celery Workers file_transfers",
            "type": "python",
            "env" : {
                "GEVENT_SUPPORT": "true"
            },
            "request": "launch",
            "module": "celery",
            "console": "integratedTerminal",
            "args": ["-A", "src", "worker", "-l", "debug", "-P", "gevent", "--concurrency" , "10", "-Q", "file_transfers",  "-E", "--hostname=file_transfers@%h"],
            "justMyCode": true
        },        
        {
            "name": "Python: Celery Beat",
            "type": "python",
            "request": "launch",
            "module": "celery",
            "console": "integratedTerminal",
            "args": ["-A", "src", "beat", "-l", "critical"]
         },
         {
            "name": "Python: Django Shell",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/manage.py",      
            "args": [
                "shell_plus",
                "--ipython"
            ],
            "django": true,
            "justMyCode": false
        }
    ]
}

Declare Environment Variables

Set secrets in the environment variables in the .bashrc file. Activate .bashrc file by running the command: source .bashrc

If using PyCharm IDE, declare the environment variables in the following places:

  • Edit Configurations > Environment variables

  • File->Settings->Build, Execution, Deployment->Console->Python console

  • File->Settings->Build, Execution, Deployment->Console->Django console

  • File->Settings->Languages & Frameworks->Django->Enable Django

Set up Python Virtual Environment

Run the following commands within the project directory to set up a Python virtual environment for the project

Alternatively, you can create a virtual environment for the project with the command: python3 -m venv venv

Install packages with pip3 and Python3 
 sudo apt install -y python3-virtualenv
 source venv/bin/activate
 pip3 install -r requirements/base.txt
 pip3 install celery
 python manage.py migrate && python manage.py social_accounts && python manage.py setup_groups && python manage.py setup_schemas && python manage.py createcachetable && supervisord -c celery.conf && supervisorctl -c celery.conf start all
 python manage.py createsuperuser

Warning

Here are solutions to possible errors that you might encounter after having ran the above commands:

  • Possible error #1: TypeError: ‘Collection’ object is not callable. If you meant to call the ‘authenticate’ method on a ‘Database’ object it is failing because no such method exists.

    Solution #1: Run the command: $ pip3 install pymongo==3.11.4 Reason: pymongo versions >4 is incompatible with the COPO project

  • Possible error #2: Issues related to importlib, rdlib, lxml, numpy, pandas and cffi Python packages

    Solution #2: #. In the requirements/base.txt file located in the project directory, change the version numbers for the following packages so that all packages can be compatible with each other

    • asgiref==3.7.2

    • cffi==1.15.1

    • importlib-metadata==1.6.1

    • lxml==4.9.3

    • numpy==1.20.0

    • pandas==1.2.2

    1. Change line 12 in wizard_helper.py file from pandas.io.json import json_normalize to from pandas import json_normalize

    2. Run the command: pip3 install -r requirements/base.txt again to install the packages

  • Possible error #2: text index required for $text query

    Solution #3: Create an index in the SampleCollection in mongoDB: db.SampleCollection.createIndex( { "$**": "text" } )


Set Up Mongo Database in COPO Project

Note

Replace the username and password for MongoDB with your username and password

Run the 4_mongo_setup.sh MongoDB set up script to configure the MongoDB database on your machine

Set up MongoDB in Studio3T

Note

Replace the <username> and <password> for MongoDB with your MongoDB username and password By default, the username for MongoDB is copo_user. The hostname,``localhost``,, can also be substituted with the IP address of 127.0.0.1

Create a new MongoDB connection manually with the following details:

  • Connection Name: copo_mongo

  • Connection Type: Standalone Connection

  • Hostname: localhost

  • Port: 27017

  • Authentication: Username and Password

  • Username: copo_user

  • Password: <password>

  • Authentication Database: admin

OR

MongoDB URI: mongodb://copo_user:password@localhost:27017/?retryWrites=true&loadBalanced=false&serverSelectionTimeoutMS=5000&connectTimeoutMS=10000&authSource=admin

Set Up PostgreSQL Database in COPO Project

Note

Replace the username and password of PostgreSQL with your username and password

Run the postgresqlDB_setup.sh PostgreSQL set up script to configure the PostgreSQL database on your machine

Launch COPO Website

Launch COPO website using the following command 
python manage.py runserver

Alternatively, click the Run button (i.e the green play button) on the top-right corner of the PyCharm IDE to launch the website.