Using the experiment queue

We recommend using the queue for long-running experiments that require no interaction.

The queue works like a compute cluster: you submit an experiment protocol, the chamber runs it when ready, and it uploads the data to a server for you to download.

We store your experimental data for the entire duration of your contract.

Basic workflow

The API is very similar to that of the real-time experiments. You can find a complete example below.

1

Connect to the Remote Lab

First, open a connection to the remote lab (see also Quickstart)

Credentials file

import causalchamber.lab as lab

rlab = lab.Lab(credentials_file = 'path/to/file')

Environment variables

2

Create a new experiment protocol

Then, start a new protocol by specifying which chamber (chamber_id) and hardware configuration (config) you want it to run on

experiment = rlab.new_experiment(chamber_id, config)

3

Add instructions

Add instructions to the experiment protocol

SET instruction

experiment.set(target, value)

This sets the variable target to the given value, returning when the change has been made in the hardware. The above call returns None.

See the configuration docs for a list of variables and their valid / default values.

MEASURE instruction

WAIT instruction

You can check which instructions are already in the protocol by calling experiment.instructions. Calling experiment.clear() will remove all instructions.

You can also generate instructions directly from a pandas dataframe (see below).

4

Submit your experiment

Once you are ready, submit the experiment to the chamber's queue with

experiment.submit(tag='optional-tag')

This will return a experiment_id that uniquely identifies your experiment in the system.

To help you keep track of your experiments, you can also add an optional tag parameter with a string of your choice (alphanumeric characters and +-_:).

Monitoring your experiments

You can check on all your current and past experiments with

rlab.get_experiments(print_max=10) # 0 for no print, None to print all

Output

You can also query the details (incl. status) for an individual experiment with

rlab.get_experiment(experiment_id)

Checking your position in the queue

You can monitor the running and queued experiments for a given chamber by calling

rlab.get_queue(chamber_id)

Output

Cancelling an experiment

You can cancel a QUEUED or RUNNING experiment by calling

rlab.cancel_experiment(experiment_id)

For RUNNING experiments, the experiment will temporarily transition to status STOPPING while the chamber finishes executing the current instruction. This may take a moment if that happens to be a long measure or wait instruction.

Downloading the data

Once an experiment is finished (status=DONE), you can download the data by calling

data = rlab.download_data(experiment_id, root='path/to/download/dir')

where root specifies the directory where you want to store the data. Then, load the data into the desired format

data.dataframe       # measurements as a pandas dataframe
data.image_arrays    # list of image arrays (H x W x 3)
data.image_iterator  # iterator over image arrays

A complete example

Let's submit an experiment to visualize the effect of Malus' law of polarization in the Light Tunnel Mk2. You can learn more about this effect in Appendix IV.2.1 of the original chambers paper.

For our experiment, we will keep the light source fixed at a constant brightness and take measurements for random polarizer positions.

import causalchamber.lab as lab
import numpy.random as random

# Connect to the remote lab
rlab = lab.Lab(credentials_file='.credentials')

# Start a new experiment protocol
experiment = rlab.new_experiment(chamber_id = 'lt-demo-ch4lu', config = 'standard')

# Add instructions
[experiment.set(color, 255) for color in ['red', 'green', 'blue']]
for i in range(100):
    # Set polarizers to random positions
    experiment.set('pol_1', random.uniform(-90,90))
    experiment.set('pol_2', random.uniform(-90,90))
    # Take one measurement
    experiment.measure(n=1)

# Submit the experiment
experiment_id = experiment.submit(tag='demo-malus')

You can monitor the experiment with rlab.get_experiments() and load the measurements into a pandas dataframe once it's done.

data = rlab.download_data(experiment_id, root='/tmp')
df = data.dataframe

Now we can plot the light intensity after both polarizers vs. their relative angle (see ir_3, pol_1, pol_2 in the configuration docs). For comparison, we show the prediction from Malus' law in red.

import matplotlib.pyplot as plt
import numpy as np

plt.figure(figsize=(8,3))

# Plot light intensity vs. relative polarizer angle
plt.scatter(df.pol_1 - df.pol_2, df.ir_3, c='gray', edgecolor='black')

# Plot Malus' law
x = np.arange(-180,180)
plt.plot(x, 92.30 + 2499.65 * np.cos(np.radians(x))**2, 'r--')

plt.legend(['Measurements', "Malus' law"], loc='upper left')
plt.xlabel('pol_1 - pol_2'); plt.ylabel('ir_3')

Generating instructions from a pandas dataframe

To make things easier when creating experiments, you can also generate instructions from a pandas dataframe by calling experiment.from_df(...) . For example, the above experiment can be rewritten as

import causalchamber.lab as lab
import numpy.random as random
import pandas as pd

# Connect to the remote lab
rlab = lab.Lab(credentials_file='.credentials')

# Start a new experiment protocol
experiment = rlab.new_experiment(chamber_id = 'lt-demo-ch4lu', config = 'standard')

# Add instructions
[experiment.set(color, 255) for color in ['red', 'green', 'blue']]
df = pd.DataFrame({
    'pol_1': random.uniform(-90, 90, size=100),
    'pol_2': random.uniform(-90, 90, size=100)
})
experiment.from_df(df)

# Submit the experiment
experiment_id = experiment.submit(tag='demo-malus')

You can find more details about how the function works (e.g., to customize the number of measurements per row) in its docstring.