Installation

Installation#

Tip

  • Don’t be discouraged if everything does not install in the first go. Use Discord (invitation link) and the TAs to get help.

Installation instructions for windows

This course supports the following python versions:

Note

All tests are known to work for the supported python versions. However, problems are often specific to certain computers/operating systems. Don’t worry, all problems should be fixable!

Step 1: Install an IDE#

Note

I would not recommend Spyder. It is both janky and underpowered.

An IDE is the tool we use to edit code. I personally use Pycharm (see Appendix A: Setting up Pycharm if you wish to use Pycharm), but VS Code is a little more popular at DTU, so therefore I will provide instructions for VS Code.

Need help?

If you run into issues following the instruction guide, you can get help from the Python support.

Python support:

pythonsupport.dtu.dk

Step 2: Install the Python extension for VS Code#

To enable Python support in VS Code, you must install the Python extension. To do this:

  • From the activity bar, click on the icon with the four boxes indicated in the screenshot.

  • This will open a pane. Type python in the search field and hit enter

  • Install the Python extension (top most search hit). It will have Microsoft as publisher.

Step 3: Get the course software and exercises#

Warning

You can put the 02465students-folder any place you like, but once you have completed the following steps you should not move or rename the folder for the remainder of the course, as this will confuse Python.

To get started, you must first download the course folder to your computer:

You can download the folder as a zip-file:

You should end up with a folder called: 02465students containing the source code.

Step 4: Install Python#

Note

I recommend Anaconda for windows. Anaconda will automatically install the program called swig (which is required for gymnasium) and that will save you some troubles. If you don’t like anaconda, you can install swig manually on windows (use google to find instructions), and you can use anaconda on macos or Linux if you prefer.

To run Python files, you must have a Python interpreter installed. The interpreter is responsible for actually running your programs.

Download and install Miniconda from https://docs.conda.io/en/latest/miniconda.html.

Important:

  • Make sure to check the option For this user only when prompted

Then next time you are prompted:

  • Make sure to check the option Add Miniconda to my PATH variable

  • Make sure to check the option Register Miniconda as my default python 3.11

Step 5: Install the course toolbox and software packages#

  • Start a new terminal using Terminal ‣ New Terminal

  • Navigate to the folder you created in the step Step 3: Get the course software and exercises step, 02465students

    • You can display the current path of the terminal by typing cd (Windows) or pwd on (Mac/Linux).

    • You change to a subfolder by typing cd name_of_the_folder, or move one folder up by typing cd ...

  • Check that you have navigated to the correct folder by verifying that it contains requirements_pip.txt.

    • You can list which files are in the folder by typing dir (Windows) or ls (Mac/Linux).

  • See Appendix C: Conda guide for more conda commands or install conda in VS Code which this guide is based on.

  • Create a new conda environment by running:

    conda create --name my_irlc_env python=3.11   # The name is up to you.

  • Select your environment in VS Code. To do this open the command pallet using ctrl+Shift+P, type Python: Select Interpreter, and select your environment (my_irlc_env) from the list. If it does not appear, select Enter interpreter path.. and navigate to it.

  • important: Open a new terminal using Terminal ‣ New Terminal

    • The new terminal should display the name of your environment my_irlc_env to the left. See step 5 of install conda in VS Code.

  • Navigate to the 02465students folder and run:

    conda install -c conda-forge --file requirements_conda.txt
pip install -r requirements_pip.txt

Check the output when you run the install commands to see if they exit with an error. Many common problems can be solved using the pythonsupport.dtu.dk

Making sure your files are up to date#

Updating your files

Since the course is being renewed, I plan to release the exercise and project code during the course. I have written a small python script to update the files, irlc/update_files.py. You can run it like a normal file, or from the terminal:

python -m irlc.update_files

Note

The file will check your version of the toolbox against the most recent version on gitlab. It will then

  • Replace any files you are missing

  • Overwrite the tests for the weekly exercises and projects

It will not overwrite your solutions to the exercises and projects. If you may have changed a file and wish to reset it, just delete it from your computer and run update_files.py to update.

Known Issues#

Even though the installation worked for the TAs and me you might experience problems specific to your system. This guide will try to keep track of common problems and their solution

Cannot find file#

You get an error such as:

ModuleNotFoundError: No module named 'irlc.exam.midterm2023a.question_pid'

First check that the file you are trying to run exists (in this case irlc/exam/midterm2023a/question_pid.py). If this is the case, you probably have two copies of the irlc-package, and the file is not located in the one you installed in Step 5: Install the course toolbox and software packages.

Solution: To check where you installed irlc, run the following command from the VS Code terminal

Note

on MacOS, assuming you normally run programs with python3, use: python3 -c"import irlc; print(irlc.__path__)"

python -c"import irlc; print(irlc.__path__)"

On my computer this will print out ['/home/tuhe/Documents/02465students/irlc']. Then make sure the file you are trying to run is located in this directory, i.e. as /home/tuhe/Documents/02465students/irlc/exam/midterm2023a/question_pid.py.

Windows#

This application failed to start because it could not find or load the Qt platform plugin “windows”

Try these commands in the VS Code terminal:

pip uninstall pyqt5
conda install pyqt

If this fail to work see this thread.

Appendix A: Setting up Pycharm#

If you wish to use Pycharm instead of VSCode, this is how I recommend you set it up.

  • Install Pycharm community edition

  • Start pycharm and create a new project. The name can be anything you like such as 02465rl

  • The root of the project can be anything – we are going to fix that later.

  • Select the interpreter you installed. If you followed the above guides, it should probably be under System interpreter

  • Create project and wait for pycharm to load. Then open settings using ctrl+alt+s (or use the file menu)

  • Under project: <projectname> select Interpreter

  • Check you got the main packages you installed such as gym. If not, select an interpreter from the list. If the python version you installed is not show, use show all from the dropdown and you can add/configure the one you installed

  • Under project: <projectname> select Project structure

  • Select Add project root. Navigate to and add the 02465students folder.

  • again in the settings dialog, under Tools select Python integrated tools. Then change Default test runner to just Unittest.

  • You are now done. You should be able to run and debug files. Note that pycharm will not allow you to run files while it is indexing your packages (check progress bar in lower right corner)

In case Pycharm runs your tests twice (and first time fails)#

Recent versions of pycharm use something called Pytest to set up and evaluate the test. Pytest has a bug where it will run the same test twice but in a different way, leading the test to crash in the first go. A way to tell this has happened is to look at the error message and see if it mentions pytest near the beginning.

The way to fix that is to disable pytest and use regular python unittest instead. This is easy:

  • Click File then Settings..

  • Click Tools and select Python integrated tools

  • From the dropbox Default test runner select Unittest (see screenshot)

Selecting your test runner.

This will mean new tests are no longer run by Pytest, but Pycharm has a habit of remembering previous run configurations so if the problem occurs again, go in the top-right corner, click the drop-down menu left of the run-button (see screenshot below) and select Edit Configurations. This will open a window with all configurations. From the menu on the left, find all configurations that mention tests, select them, and press the small minus sign to delete them – don’t worry if you delete one too many, pycharm will recreate them. (see screenshot)

Deleting bad configurations.

Appendix B: Python 3.12#

Python 3.12 is the most recent version of python. It is not yet officially supported by gymnasium, pygame and pytorch, and therefore I recommend that you use python 3.11. If you still wish to use Python 3.12, you should first install the pre-release version of pytorch:

pip install --pre torch --index-url https://download.pytorch.org/whl/nightly/cpu

and then follow the installation instructions for your system. Using this approach I can run all tests on python 3.12/Linux, but you might experience platform-specific problems where the only solution is to wait for python 3.12 support.

Appendix C: Conda guide#

Tip

Learn from my mistakes and get in the habit of using conda now and not later. It will save you hours down the line

When starting a new project or course it is always recommended to create a new virtual environment. We do this to prevent dependency issues down the line when installing new packages.

To create a new environment with a specific python version, do the following:

conda create -n <env_name> python=<version>

Recommended version <version>=3.10 Pick an appropriate name for the environment like: <env_name>=rl_control

Use the below command to see all current anaconda environments and where they are installed. The currently activated environment is marked with a *

conda env list

Then activate the newly created environment

conda activate env_name

In your terminal you should now see (env_name) in front of your current file path. The name given in the parenthesises are indicating the currently activated environment and is also the environmnet where packages will be installed when using conda (recommended) or pip.

You can now continue with the provided installation guide.

For vscode users#

To ensure that vscode uses the correct virtual conda environment you should always launch vscode from within the anaconda navigator (terminal). This is achieved by running <code>code</code> in the terminal.

code

If you wish to open vscode within a specific folder you can run

code <path>

Use code . to launch vscode in the current working directly.

Deleting environments#

Note

Thanks to Calle for contributing this guide!

Since we are installing complete sets of libraries every time we create a new environment over time we may have a bunch of old environment taking up unnecessary space. To delete an environment run the following code:

conda env delete <env_name>

If for some reason this command fails you can also delete the folder manually. Remember that you can use conda env list to show where conda has installed the environment.