Examples

You can find these examples in the examples directory (surprise!) of the Sacred sources or in the Github Repository. Look at them for the sourcecode, it is an important part of the examples. It can also be very helpful to run them yourself and play with the command-line interface.

The following is just their documentation from their docstring which you can also get by running them with the -h, --help or help flags.

Hello World

examples/01_hello_world.py

This is a minimal example of a Sacred experiment.

Not much to see here. But it comes with a command-line interface and can be called like this:

$ ./01_hello_world.py
WARNING - 01_hello_world - No observers have been added to this run
INFO - 01_hello_world - Running command 'main'
INFO - 01_hello_world - Started
Hello world!
INFO - 01_hello_world - Completed after 0:00:00

As you can see it prints ‘Hello world!’ as expected, but there is also some additional logging. The log-level can be controlled using the -l argument:

$ ./01_hello_world.py -l WARNING
WARNING - 01_hello_world - No observers have been added to this run
Hello world!

If you want to learn more about the command-line interface try help or -h.

Hello Config Dict

examples/02_hello_config_dict.py

A configurable Hello World “experiment”. In this example we configure the message using a dictionary with ex.add_config

You can run it like this:

$ ./02_hello_config_dict.py
WARNING - 02_hello_config_dict - No observers have been added to this run
INFO - 02_hello_config_dict - Running command 'main'
INFO - 02_hello_config_dict - Started
Hello world!
INFO - 02_hello_config_dict - Completed after 0:00:00

The message can also easily be changed using the with command-line argument:

$ ./02_hello_config_dict.py with message='Ciao world!'
WARNING - 02_hello_config_dict - No observers have been added to this run
INFO - 02_hello_config_dict - Running command 'main'
INFO - 02_hello_config_dict - Started
Ciao world!
INFO - 02_hello_config_dict - Completed after 0:00:00

Hello Config Scope

examples/03_hello_config_scope.py

A configurable Hello World “experiment”. In this example we configure the message using Sacreds special ConfigScope.

As with hello_config_dict you can run it like this:

$ ./03_hello_config_scope.py
WARNING - hello_cs - No observers have been added to this run
INFO - hello_cs - Running command 'main'
INFO - hello_cs - Started
Hello world!
INFO - hello_cs - Completed after 0:00:00

The message can also easily be changed using the with command-line argument:

$ ./03_hello_config_scope.py with message='Ciao world!'
WARNING - hello_cs - No observers have been added to this run
INFO - hello_cs - Running command 'main'
INFO - hello_cs - Started
Ciao world!
INFO - hello_cs - Completed after 0:00:00

But because we are using a ConfigScope that constructs the message from a recipient we can also just modify that:

$ ./03_hello_config_scope.py with recipient='Bob'
WARNING - hello_cs - No observers have been added to this run
INFO - hello_cs - Running command 'main'
INFO - hello_cs - Started
Hello Bob!
INFO - hello_cs - Completed after 0:00:00

Captured Functions

examples/04_captured_functions.py

In this example the use of captured functions is demonstrated. Like the main function, they have access to the configuration parameters by just accepting them as arguments.

When calling a captured function we do not need to specify the parameters that we want to be taken from the configuration. They will automatically be filled by Sacred. But we can always override that by passing them in explicitly.

When run, this example will output the following:

$ ./04_captured_functions.py -l WARNING
WARNING - captured_functions - No observers have been added to this run
This is printed by function foo.
This is printed by function bar.
Overriding the default message for foo.

My Commands

examples/05_my_commands.py

This experiment showcases the concept of commands in Sacred. By just using the @ex.command decorator we can add additional commands to the command-line interface of the experiment:

$ ./05_my_commands.py greet
WARNING - my_commands - No observers have been added to this run
INFO - my_commands - Running command 'greet'
INFO - my_commands - Started
Hello John! Nice to greet you!
INFO - my_commands - Completed after 0:00:00

$ ./05_my_commands.py shout
WARNING - my_commands - No observers have been added to this run
INFO - my_commands - Running command 'shout'
INFO - my_commands - Started
WHAZZZUUUUUUUUUUP!!!????
INFO - my_commands - Completed after 0:00:00

Of course we can also use with and other flags with those commands:

$ ./05_my_commands.py greet with name='Jane' -l WARNING
WARNING - my_commands - No observers have been added to this run
Hello Jane! Nice to greet you!

In fact, the main function is also just a command:

$ ./05_my_commands.py main
WARNING - my_commands - No observers have been added to this run
INFO - my_commands - Running command 'main'
INFO - my_commands - Started
This is just the main command. Try greet or shout.
INFO - my_commands - Completed after 0:00:00

Commands also appear in the help text, and you can get additional information about all commands using ./05_my_commands.py help [command].

Randomness

examples/06_randomness.py

This example showcases the randomness features of Sacred.

Sacred generates a random global seed for every experiment, that you can find in the configuration. It will be different every time you run the experiment.

Based on this global seed it will generate the special parameters _seed and _rnd for each captured function. Every time you call such a function the _seed will be different and _rnd will be differently seeded random state. But their values depend deterministically on the global seed and on how often the function has been called.

Here are a couple of things you should try:

• run the experiment a couple of times and notice how the results are different every time

• run the experiment a couple of times with a fixed seed. Notice that the results are the same:

  :$ ./06_randomness.py with seed=12345 -l WARNING
  [57]
  [28]
  695891797
  [82]
  

• run the experiment with a fixed seed and vary the numbers parameter. Notice that all the results stay the same except for the added numbers. This demonstrates that all the calls to one function are in fact independent from each other:

  :$ ./06_randomness.py with seed=12345 numbers=3 -l WARNING
  [57, 79, 86]
  [28, 90, 92]
  695891797
  [82, 9, 3]
  

• run the experiment with a fixed seed and set the reverse parameter to true. Notice how the results are the same, but in slightly different order. This shows that calls to different functions do not interfere with one another:

  :$ ./06_randomness.py with seed=12345 reverse=True numbers=3 -l WARNING
  695891797
  [57, 79, 86]
  [28, 90, 92]
  [82, 9, 3]
  

Less magic

If you are new to Sacred, you might be surprised by the amount of new idioms it introduces compared to standard Python. But don’t worry, you don’t have to use any of the magic if you don’t want to and still benefit from the excellent tracking capabilities.

examples/07_magic.py shows a standard machine learning task, that uses a lot of possible Sacred idioms:

• configuration definition through local variables
• parameter injection through captured functions
• command line interface integration through the ex.automain decorator

examples/08_less_magic.py shows the same task without any of those idioms. The recipe for replacing Sacred magic with standard Python is simple.

• define your configuration in a dictionary, alternatively you can use an external JSON or YAML file
• avoid the ex.capture decorator. Instead only pass _config to the main function and access all parameters explicitly through the configuration dictionary
• just use ex.main instead of ex.automain and call ex.run() explicitly. This avoids the parsing of command line parameters you did not define yourself.

While we believe that using sacred idioms makes things easier by hard-wiring parameters and giving you a flexible command line interface, we do not enforce its usage if you feel more comfortable with classical Python. At its core Sacred is about tracking computatonal experiments, not about any particular coding style.

Docker Setup

examples/docker

To use Sacred to its full potential you probably want to use it together with MongoDB and dashboards like Omniboard that have been developed for it. To ease getting started with these services you find an exemplary docker-compose configuration in examples/docker. After installing Docker Engine and Docker Compose (only necessary for Linux) go to the directory and run:

docker-compose up

This will pull the necessary containers from the internet and build them. This may take several minutes. Afterwards mongoDB should be up and running. mongo-express, an admin interface for MongoDB, should now be available on port 8081, accessible by the user and password set in the .env file (ME_CONFIG_BASICAUTH_USERNAME and ME_CONFIG_BASICAUTH_PASSWORD). Sacredboard ``should be available on port ``5000. Omniboard should be available on port 9000. They will both listen to the the database name set in the .env file (MONGO_DATABASE) which will allow the boards to listen to the appropriated mongo database name set when creating the MongoObserver with the db_name arg. All services will by default only be exposed to localhost. If you want to expose them on all interfaces, e.g. for the use on a server, you need to change the port mappings in docker-compose.yml from 127.0.0.1:XXXX:XXXX to XXXX:XXXX. However, in this case you should change the authentication information in .env to something more secure.