Tutorial Lesson 3: input/output

• input parameters on the command-line or from a text file
• save the population to disk, together with every part of the algorithm you could think of - so you can decide to reload everything later to continue the same run, eventually with different parameters.
• generate statistics on the populations, and output them to the screen, text or graphic, or to a file (or to any other device you might want to use).

You can then either

• browse the corresponding code (SecondBitEA and SecondRealEA),
• look at the summary of changes,
• or find out directly explanations about the new features: the eoParser, eoState and eoCheckpoint classes.

• Fitness function: there is an additional line after the encapsulation of our binary_function into an eoEvalFunc object, which again encapsulate the eoEvalFunc into an eoEvalFuncCounter. As its name says, this object will, in addition to computing the fitness, count the actual number of evaluations: the fitness of non-modified individuals is of course not recomputed - and this is taken care of by this object. Moreover, it can be later used for displays in eoMonitor objects, as done in the checkpoint section.
• The initialization section has been extended to account for the possibility to re-load a previously saved population. This is achieved through an eoState object, if the corresponding program parameter is set.
• The variation operators and the evolution engine  sections are similar to the ones in Lesson2
• The parameter section  is completely different from the previous one. All variables corresponding to program parameters are now read at run-time using an object of class eoParser.
• The stopping criterion section, has in fact now become the checkpoint section, as it involves much more than just stopping criteria. See all details in the eoCheckpoint paragraph below.

• The eoValueParam class, templatized by the type of the variable you want to handle (i.e. iinteger, double, yourPrivateClass, ...). In this lesson, we will not go into details: e.g. we will not tell you that the eoValueParam is actually a templatized sub-class of abstract class eoParam (oops, I said it!), nor will we deal with parameters outside their use from an eoParser. See the parameter section of the Component-Based tutorial, or wait until lesson 4).
• The eoParser class, whose only purpose is the input of parameters. Read its description if you are interested.

1. from the command-line
2. from a text file
3. from the environment (forthcoming, if somebody insists)
4. from default values

• in EO, each parameter is designated by a (long) keyword, and optionally by a short (1 character) keyword.

 
• the general syntax to modify parameter value at run-time is (either from the command-line or in a text file)

             --longKeyword=value     or     -cvalue    if 'c' is the short keyword (though -c=value also works)
 
• so, after compiling the executable for Lesson 3 (make lesson3 at system prompt in Unix), you can try to type in

              SecondBitEA
and see the algorithm run as before (OneMax optimized on 8-bits bitstrings). But you can now type in
              SecondBitEA --vecSize=100
and see the output of the optimization of OneMax on 100-bit bitstrings.
 
• Take a look at all available parameters by typing in

              SecondBitEA --help
or by going into the code: all parameter inputs have been grouped in the read_param function.
 
• After running the algorithm, a new file has been created, named SecondBitEA.status: it contains the list of all actual parameters used, and can directly be used as parameter input file: change the file name (e.g. to SecondBitEA.param), edit it, change whichever parameter you want, and type in

                SecondBitEA @SecondBitEA.param
and you will see all values that you defined into the file taken into account.
 
• The priority remains to the command-line, so you can still override the values in the parameter file by giving a new value directly on the command-line.

• You must first declare a parameter of type uint32 (32-bits integer). The arguments are: default value, long keyword, comment (that will appear in the help message and in the output "status" file if any) and optional character keyword.
• Then you must pass it to the parser using the processParam method. The optional argument is a section name, that will be used to make the output of the parser look clean and ordered.
• Finally, you need to assign the value to the variable seed. Note that the value() method of eoParam returns a reference, so you can eventually modify its value somewhere else later (though of course this is not any useful for variable seed!).

This is made possible using the eoState class. eoState objects maintain references to eoObjects that have both an input method (readFrom) and an output method (printOn), i.e. that derive from the base class eoPersistent. You must first register object into a state, and can then save them to a (text) file, and later read them from that file using the load method, as done here.
Of course, you can call the save method for an eoState object anywhere in the code. But the checkpointing mechanism offers you better ways to do that - and it's so easy ....

Note that an eoState alos has another use in EO whan it comes to memory management: it can be a repository of pointers that are not allocated within obects - allowing to delete them by simply deleting the eoState (see Lesson 4).

eoCheckpoint objects are eoContinue objects that contain pointers to different types of objects. When their operator() method is called (i.e. every generation in the examples up to now), they first call the operator() methods of all object they contain, and then return their result as an eoContinue object (i.e. should we continue or stop).
Programming: To do something every generation, you simply need to add an object whose operator() does what you want to the eoState that you will use as continuator in the algorithm.

Statistics: Available instances
Some widely used statistics are already available (and of course you can build you own!).

• eoBestFitnessStat returns the fitness value of the best individual in the population (of type FitnessType, whatever this is).
• eoAverageStat and eoSecondMomentStat respectively return the average (type double, assumes that FitnessType is castable to a double) and a pair made of the average and the standard deviation (type pair<double>) of the fitnesses in the populations.
• eoDiversityStat returns the diversity in the population: assuming that there is a distance function defined among individuals, it returns the average inter-individuals distance. See also Exercise 2.

Note: actually, there are 2 distinct classes that compute and give access to statistics: eoStatand eoSortedStat. As its name indicate, the latter is used whenever computing the statistics require a sorted population: not only this avoids to sort the population many times, but also it avoids changing the order of the population at all as eoSortedStat objects work on a temporary vector of fitnesses . But as far as their usage is concerned, its makes no difference.

Monitors: Available instances
A few monitors are available in the EO distribution:

• eoStdoutMonitor displays its parameters in text format on the screen. The (optional) boolean value in the constructor modifies the output: when true (the default), verbose output is used, with one line per parameter. When false, parsimonious output displays one line for all parameters.
• eoFileMonitor writes its parameters in text format in a file. A file name is required in the constructor, and an optional separator character can be added (default is ' '). Note that the file is by default overwritten by next call to the same program, unless you pass "true" as third (optional) boolean parameter, which will result in appending to the file if it ever exists.
• eoGnuplot1DMonitor displays its parameters in graphical format on the screen by calling the gnuplot program, and as of today, only works in the Unix version of EO (as always, volunteers are welcome to port that to MS Windows). It takes an optional filename as input, as communication of data with gnuplot is done through a file. If no filename is provided, the file will be erased at the end of the run, while it is otherwise kept (though it will be overwritten by next call to the same program).

Monitors: Adding to the checkpoint
To display something while the algorithm is running, you need to declare an eoMonitor object, add some objects (that must be eoValueParam objects) to that monitor, and of course add the monitor to the eoCheckpoint you use in the algorithm.

Updater: Available instances: A few updaters are available in the EO distribution:

• eoIncrementor A simple updater which maintains a counter (an eoValueParam that needs to be created beforehand, and passed in the constructor). It is incremented every time the operator() method is called (every generation at the moment). You can of course also give an increment in the constructor (1 by default).
• eoCountedStateSaver and eoTimedStateSaver can be used to save some existing eoState (see above) to a file regularly, either based on the generation count (e.g. every 4 generations) or based on the clock (e.g. every 5 seconds).

• The code of SecondBitEA display things in the current window in text format. Replace the eoFileMonitor by an eoGnuplot1DMonitor and watch the graphical output (Unix systems with gnuplot installed only, sorry).
• Note that you must also replace the eoSecondMomentStat by an eoAverageStat, otherwise the standard deviations won't make any sense here.
• Please try to understand why the average is always 0 before taking a look at the solution (file exercise1.cpp).
• Then run

          exercise1 --vecSize=1000 --maxGen=1000
to get a chance to see something happening before the program ends!

• Value of program parameters can be set at run-time using the eoParser class.
• Snapshots of the algorithms can easily be saved (and restored) thanks to the eoState class.
• The eoCheckpoint mechanism let you do things every generation without modifying existing algorithms, by simply writing the necessary code and encapsulating it into an object that eoCheckpoint is aware of, that are at the moment the following:
• computing statistics, displaying parameters (e.g. statistics),  saving the (eo)State of the program.