GENESIS: Documentation

Related Documentation:

• GENESIS System
• Neurospaces Tester

System Administration Log

This is a document listing all of all administration done on machines used in the GENESIS project.

Please only add records. Do not delete or modify existing records.

In the past the file system of some of the machines supporting for G-3 development had become corrupted because of all the manual maintenance. Therefore we keep track of the manual maintenance, and try to automate as much as possible over the source code repositories and cron jobs.

repo-genesis3.cbi.utsa.edu

The repository machine serves the source code for the GENESIS project as well as periodically performing automated source code checks. This machine also has a running web server that hosts the projects: doxygen documentation, regression tests, and various package distributions.

System specs:

• Processor: 2 Dual-Core AMD Opteron(tm) Processor 2220.
• Memory: 8 GB
• Operating System: Debian Linux
• Monotone version: monotone 0.48 (base revision: 844268c137aaa783aa800a9c16ae61edda80ecea)
• Python Version: 2.5.2

• • Admin: Mando
  • Date: 5/26/2012
  • Actions: Checked kernel and other system logs for any errors.
  • Reason: Was informed by David that the repo server has a warning light flashing in the server room. Doesn’t appear to be affecting any of the cron jobs or other functions.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 4/6/2012
  • Actions: Installed the Mo::builder perl module
  • Reason: Possibly deleted this by mistake after removing some junk perl modules that were unreachable.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 4/5/2012
  • Actions: Performed calls to the perl config to see the values of the installsitelib, and siteprefix variables in the perl Config module.
  • Reason: Fedora seems to have an incompatible site prefix but it works fine on Debian/Ubuntu.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 3/21/2012
  • Actions: Reactivated package builds.
  • Reason: Packaging was disabled previously due to an error with reverting the tag updates.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 3/20/2012
  • Actions: Performed a revert across all packages in the tester project directory. Disabled all package building.
  • Reason: Discovered that the neurospaces_revert was not working correctly because it wasn’t changing directories. After fixing the script a proper revert was performed.
  • Impact: Tester should proceed as normal, will monitor for any other errors. If no other errors occur the package creation will be reenabled.
• • Admin: Mando
  • Date: 3/19/2012
  • Actions: Checked on the issue of a conflict in the model container workspace. Performed no actions.
  • Reason: An workspace conflict occurred in the model container and was reported by Hugo. The cron run at noon performs a tag update using the current date, as well as a neurospaces_revert at the tail end of its run, which seems to have ”fixed” the workspace for the next cron run. Will need to reference a log file from a previous cron run to see if I can determine where the error occurs.
  • Impact: Error resolved itself on this run due to the neurospaces_revert, still unsure of what caused the tag update to report an error. Will check the tag operations on the model container, previous cron runs and monitor for the error if it reappears.
• • Admin: Hugo
  • Date: 3/19/2012
  • Actions: Checking why there are conflicts in the model-container workspace of the tester account. Turns out that the version numbers have changed, eg.:

    ============================================================   --- Makefile.am 23764f4e166391a2e729ae52a3d864bb52de1194   +++ Makefile.am 424bec0e0f07d370a46c425183abe0a494f0f493   @@ -392,7 +392,7 @@ internal-library:      internal-library:    # $Format: "   tar cfz library-${package}-${label}.tar.gz library/"$   -       tar cfz library-model-container-alpha.tar.gz library/   +       tar cfz library-model-container-3.18.112.tar.gz library/

    I assume Mando was / is testing something, nothing done.

  • Reason: Tester reports conflicts in the model-container workspace and cannot proceed.
  • Impact: Nothing resolved, no effect, first mailing with Mando.
• • Admin: Mando
  • Date: 2/22/2012
  • Actions: Deleted some garbage files created when checking the errors with the python modules.
  • Reason: Just ensuring that the files do not remain and cause any future problems.
  • Impact: No effect.
• • Admin: Mando
  • Date: 2/21/2012
  • Actions: Performed some installed and tests under the ’mando’ account on the tester.
  • Reason: Discovered that python is installing script only modules without the SWIG compiled code to /usr/local/lib/python2.5/site-packages, while simultaneously installing complete modules to /usr/lib/python2.5/site-packages. This is causing an import conflict for python tests that run from the installed location; i.e all sspy tests. It was also leaving a lot of egg-info files in the /usr/lib/python2.5/site-packages directory.
  • Impact: Currently updating some of the book keeping in the uninstall scripts to make sure that all old meta data is removed. Also fixing the setup scripts to ensure that the module is only installed once.
• • Admin: Mando
  • Date: 2/21/2012
  • Actions: Performed some tests on the tester to see why some python imports are broken.
  • Reason: Python setup.py is no longer installing the SWIG compiled .so modules. Reason is not yet known.
  • Impact: Currently working on a fix.
• • Admin: Mando
  • Date: 1/11/2012
  • Actions: Performed a delete and revert on the configure.ac file in the userdocs directory.
  • Reason: A request for a merge was refusing to update the workspace. Fixing locally and syncing doesn’t seem to get rid of the problem so a manual intervention was needed.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 12/15/2011
  • Actions: Killed around 80 open office processes that were left in memory along with 12 zombie processes for neurospaces_serve. Added a cron script that performs a killall on all soffice.bin processes just before the userdocs system performs its cron run. As well as a cron script that will kill all zombie processes daily.
  • Reason: The large amount of processes lying around was causing out of memory errors during some cron runs.
  • Impact: Tester should proceed as normal, will monitor for any problems.
• • Admin: Mando
  • Date: 12/14/2011
  • Actions: Reset permissions on the model container output directory.
  • Reason: The output directory was owned by root. Not sure why this happened, other tester was fine and there are no erroneous commands in the bash history. Will monitor for any changes.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 11/16/2011
  • Actions: Installed docutils and unrtf on the tester.
  • Reason: Allows the processing of rtf and rst files..
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 11/15/2011
  • Actions: Installed openoffice and all of the modules needed for headless running on the repo. Added a python script executable for running jodconverter.
  • Reason: The userdocs mirror has an error preventing processing of word docs.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 11/14/2011
  • Actions: Installed gcj and jobconverter.
  • Reason: Allow the processing of word docs.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 9/14/2011
  • Actions: Added a ’python_eggs’ directory to be accessible from the web and checked the current version of python.
  • Reason: PyPi allows secondary download sites so I’ll link the pypi download to the repositories ’python_eggs’ directory.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 8/11/2011
  • Actions: Removed user from the tester email list.
  • Reason: Received a request from the user to remove their email from the tester status notifications.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 8/10/2011
  • Actions: Performed a revert on the heccer and sspy directories in the g3tester user.
  • Reason: A tag update caused a conflict that had to be resolved with a manual merge similar to the error that previously occurred.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 8/09/2011
  • Actions: Performed a revert on the experiment directory.
  • Reason: The files that have release tags updated have not been reverted and are causing the tester to ask for a manual merge. Seems that the neurospaces_revert that is performed during distribution production did not revert the repository into a safe state.
  • Impact: Tester should proceed as normal.
• • Admin: Hugo
  • Date: 7/17/2011
  • Actions: Started each monotone server manually.
  • Reason: repo and other machines were moved to a separate rack in the server room, see entry below.
  • Impact: All repositories are up, it looks like the website is still down, tester should be ok, not checked.
• • Admin: Mando
  • Date: 7/17/2011
  • Actions: Started the monotone server
  • Reason: repo and other machines were moved to a separate rack in the server room
  • Impact: Tester started 4 repositories but the rest did not start automatically via the neurospaces_serve script.
• • Admin: Hugo
  • Date: 7/16/2011
  • Actions: Checking monotone public keys and write-permission file.
  • Reason: Checking monotone public keys and write-permission file.
  • Impact: None.
• • Admin: Hugo
  • Date: 7/9/2011
  • Actions: As root:

    cd ~/neurospaces_project/MTN/   rm chemesis3.mtn   cd ../   rm -fr chemesis3/   cd   neurospaces_upgrade --regex chemesis3

  • Reason: Checking whether upgrades work with chemesis3.
  • Impact: Chemesis3 hopefully configured correctly, regression tests should be included in the test runs. Server still in Belgium.
• • Admin: Hugo
  • Date: 7/9/2011
  • Actions: As root:

    neurospaces_packages                        <------ shows chemesis3 NOT included in package list, developer package not updated yet   neurospaces_pull --regex developer   neurospaces_update --regex developer   neurospaces_install --regex developer   neurospaces_packages                        <------ shows chemesis3 included in package list   neurospaces_init --regex chemesis3          <------ shows where chemesis3 repo is placed   neurospaces_create_directories chemesis3    <------ shows where chemesis3 is created   neurospaces_build --help-repo               <------ shows server monotone configuration

  • Reason: Initialization of the chemesis3 package by first pushing from my laptop, then installing it on repo-genesis3.
  • Impact: Chemesis3 hopefully configured correctly, regression tests should be included in the test runs. Server still in Belgium.
• • Admin: Mando
  • Date: 6/27/2011
  • Actions: Started the monotone server
  • Reason: Server was down following a blown fuse in the server room.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 5/19/2011
  • Actions: Restarted all monotone servers.
  • Reason: Was receiving a connection failure for the developer package in when trying to sync or pull.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 5/19/2011
  • Actions: Deleted the glue/swig/python/neurospaces/heccer/__cbi__.py and performed a revert to restore it to the same state as it is in the repository.
  • Reason: The rule for updating release tags was not updating correctly on linux due to a formatting error. This format error was fixed along with an update of all tags on the last checkin. The initial update created a workspace conflict with the release tags so manual intervention was needed.

    mtn: conflict: content conflict on file ’glue/swig/python/neurospaces/heccer/__cbi__.py’   mtn: content hash is 9b0cb661ff5c45ea33bd3226bf3001b25041b87f on the left   mtn: content hash is 1be6eea855af25a63ce0573020cecd8bd44297f2 on the right   mtn: help required for 3-way merge   mtn: [ancestor] glue/swig/python/neurospaces/heccer/__cbi__.py   mtn: [    left] glue/swig/python/neurospaces/heccer/__cbi__.py   mtn: [   right] glue/swig/python/neurospaces/heccer/__cbi__.py   mtn: [  merged] glue/swig/python/neurospaces/heccer/__cbi__.py

  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 5/3/2011
  • Actions: Checked the cron log and tester workspaces. Performed an install of the developer package and executed neurospaces_cron just once to ensure that it starts properly.
  • Reason: Tester has failed to send confirmation emails twice. Log file shows (with some annotations to show what happened where):

    May  2 13:59:21 repo /usr/sbin/cron[4790]: (CRON) STARTUP (fork ok)        <--( This is the cron restart from 5/2/2011 in the afternoon)   May  2 13:59:21 repo /usr/sbin/cron[4790]: (CRON) INFO (Skipping @reboot jobs -- not system startup)   May  2 14:17:01 repo /USR/SBIN/CRON[5278]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)   May  2 15:17:01 repo /USR/SBIN/CRON[5552]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)   May  2 16:01:01 repo /USR/SBIN/CRON[5839]: (root) CMD (sudo -H -u g3tester perl /home/g3tester/bin/neurospaces_cron --config /home/g3tester/neurospaces_cron.yml)  <---( Tester appears to have gone off successfully )   May  2 16:17:01 repo /USR/SBIN/CRON[5846]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)   May  2 17:17:01 repo /USR/SBIN/CRON[6157]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)   May  2 18:10:01 repo /USR/SBIN/CRON[6466]: (root) CMD (sudo -H -u g3tester perl /home/g3tester/neurospaces_project/userdocs/source/snapshots/0/bin/userdocs_cron --config /home/g3tester/userdocs_cron.yml)  <---( Tester appears to have gone off successfully )   May  2 18:17:01 repo /USR/SBIN/CRON[6479]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)   May  2 19:17:01 repo /USR/SBIN/CRON[6489]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)   May  2 20:01:01 repo /USR/SBIN/CRON[6498]: (root) CMD (sudo -H -u g3tester perl /home/g3tester/neurospaces_project/developer/source/snapshots/0/bin/neurospaces_cron --config /home/g3tester/neurospaces_cron.yml)  <---( Tester appears to have gone off successfully )   May  2 20:17:01 repo /USR/SBIN/CRON[6811]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)   May  2 21:17:01 repo /USR/SBIN/CRON[7010]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)   May  2 22:17:01 repo /USR/SBIN/CRON[7550]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)   May  2 23:17:01 repo /USR/SBIN/CRON[7569]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)   May  3 00:00:23 repo /usr/sbin/cron[7917]: (CRON) INFO (pidfile fd = 3)   May  3 00:00:23 repo /usr/sbin/cron[7919]: (CRON) STARTUP (fork ok) <--( checked and reloaded cron daemon )   May  3 00:00:23 repo /usr/sbin/cron[7919]: (CRON) INFO (Skipping @reboot jobs -- not system startup

    Tester would not proceed because I performed a neurospaces_uninstall upon exiting. This left the tester in another state in which it could not proceed, as it needed neurospaces_build installed and in the executable path.

  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 5/2/2011
  • Actions: Restarted cron daemon and checked the cron functionality with a wall test.
  • Reason: Tester appears to not have gone off at noon. The output file for neurospaces_cron has a timestamp from 8:31 am and was completely untouched. No error email was sent either.
  • Impact: None, but will closely monitor the scheduled jobs after verifying cron is working correctly.
• • Admin: Mando
  • Date: 5/2/2011
  • Actions: Performed a neurospaces_revert and a neurospaces_update.
  • Reason: Tester became ”stuck” due to a sequence of checkins that put the tester in a unusable state:

    Timeline went as follows:

    • 1. Performed some file and directory renames on the model container.
    • 2. Generated files that are cleaned during a ’make clean’ were checked in (glue/swig/python/model_container_wrap.c and glue/swig/python/neurospaces/model_container/model_container_base.py).
    • 3. Changes were synced.
    • 4. Cron run was executed. This resulted in an error since the ’make clean’ deleted the two files that were accounted for in the monotone workspace. In this state an update cannot be performed on the workspace.
    • 5. Fix with revision 32b2db358001f7fb5b8402766fa4e74bc77cba8e was checked in and synced.
    • 6. Monotone refuses to perform an update without the cleaned files present. As a result the workspace was current to revision 6d1b507fcb73f8210c4f1e2abd27c34ec9d37314.
    • 7. Tester is now stuck in an unusable state.

    The actions performed ensure that the workspace is updated to the current revision.

  • Impact: Tester should proceed as normal.
• • Admin: Hugo
  • Date: 5/1/2011
  • Actions: General integrity checking, listing directories, checking file contents and filestamps.
  • Reason: Monotone on the tester reports an error when trying to update to revision 32b2db358001f7fb5b8402766fa4e74bc77cba8e. This revision drops swig generated files, maybe this interferes with the clean target of the makefile.
  • Impact: None.
• • Admin: Mando
  • Date: 4/29/2011
  • Actions: Checked monotone version as well as checked the contents of the /usr/local/glue/swig/python directory.
  • Reason: Monotone on the tester seems to give an error when trying to update after directories containing files have been moved. This does not happen on machines that have version 99.1 installed. The /usr/local/glue/swig/python directory is the current destination for the python modules. On the mac tester case is not respected so you can’t have ’neurospaces’ and ’Neurospaces’ in the same directory. The tester respects case so no intervention is needed.
  • Impact: None.
• • Admin: Mando
  • Date: 3/1/2011
  • Actions: Checked to see if the workspace for userdocs on the tester updated to the current revision without incident.
  • Reason: Error occurred on Hugo’s laptop in userdocs that removed a file that was accounted for from the workspace, when updating to the latest revision.
  • Impact: None, appears that the repo updated without any issues.
  • Related: Ubuntu Vm on my laptop had an issue updating to the current revision and gave the error:

    mtn: 2 heads on branch ’0’   > mtn: merge 1 / 1:   > mtn: calculating best pair of heads to merge next   > mtn: [left]  435c678dbd0387be42a1d1c03f09d0d83d11092d   > mtn: [right] 618d318e72ed89230e8b8d7bb407f198e677b50b   > mtn: fatal: error: roster.cc:2086:   > I(right_uncommon_ancestors.find(right_rid) !=   > right_uncommon_ancestors.end())   > mtn: this is almost certainly a bug in monotone.   > mtn: please send this error message, the output of ’mtn version --full’,   > mtn: and a description of what you were doing to monotone-devel@nongnu.org.   > mtn: wrote debugging log to   > /home/mando/neurospaces_project/userdocs/source/snapshots/0/_MTN/debug

    After updating monotone from 0.47 to 0.99 and migrating all databases to the new schema, monotone refuses to update to any revision past the large merge node at revision 0ba094b646210d9bfb93142d82813cdb1bb80cbb when performing a userdocs-sync. An ’mtn log –last 1’ reports 0ba094b646210d9bfb93142d82813cdb1bb80cbb as its last revision. There are two heads.

    mtn: branch ’0’ is currently unmerged:   3a1235db8bfe3d46c036bd6019bbacbffec3bbaf mandorodriguez@gmail.com 03/01/2011 04:57:39 PM   618d318e72ed89230e8b8d7bb407f198e677b50b mandorodriguez+Ubuntu.Vm.laptop.i686@gmail.com hugo.cornelis@gmail.com 02/28/2011 04:43:11 PM 02/28/2011 03:41:04 PM

    Performing a ’mtn update’ without using userdocs-sync or developer scripts results in a successful merge.

• • Admin: Hugo
  • Date: 2/26/2011
  • Actions: Testing with sendmail (because it does not work on my laptop).
  • Reason: Does not work on my laptop.
  • Impact: None.
• • Admin: Mando
  • Date: 2/2/2011
  • Actions:
    • Checked the mercruial repository for errors using the ”hg verify” command.
    • Added a hugo user so that Hugo can perform a simple ”hg push” using the existing ssh key.
  • Reason:
    • Some older revisions were still on my laptop and never got pushed due to a corrupted repository, and poor error reporting from the older version of mercurial used before the laptop reinstall. The error messages looked like it was a problem with the server but it was the server rejecting the corrupted differences from the client machine.
    • Hugo saved a change in the gtube. Mercurial is set up to use ssh keys so an account makes it easier to push revisions using the same group, named hg.
  • Impact: N/A
• • Admin: Mando
  • Date: 2/1/2011
  • Actions: Added new ssh keys for passwordless access.
  • Reason: Recent reinstall of Mac OS X on my laptop was causing constant dialogs to pop up asking for a passphrase. Only way to get rid of them was to create new keys since the machine hostname was now different.
  • Impact: N/A
• • Admin: Mando
  • Date: 1/31/2011
  • Actions: Added Allans new monotone key to the userdocs repository.
  • Reason: Allan performed a reinstall on his new laptop and needed to generate a new monotone key.
  • Impact: Allan now has access to the userdocs repository. Tester should proceed as normal.
• • Admin: Mando
  • Date: 1/28/2011
  • Actions: Activated the sspy repository. Installed PyYaml.
  • Reason: Need to start syncing code between machines.
  • Impact: Tester should proceed as normal. Will monitor for any workspace issues.
• • Admin: Mando
  • Date: 1/18/2011
  • Actions: Deactivated sspy repository.
  • Reason: Automake targets are not completed, causing an error on the tester.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 1/18/2011
  • Actions: Restarted the monotone server with the sspy repository set.
  • Reason: To allow code syncing for sspy when it’s ready.
  • Impact: Developer scripts should be able to connect to the sspy repository if the option is set.
• • Admin: Mando
  • Date: 1/18/2011
  • Actions: Checked repository access to the gtube mercurial repository.
  • Reason: The mercurial repository is refusing to allow a check in after dropping a file. Getting this exception:

    bort: empty or missing revlog for genesis3/.DS_Store   remote: ** unknown exception encountered, details follow   remote: ** report bug details to http://mercurial.selenic.com/bts/   remote: ** or mercurial@selenic.com   remote: ** Python 2.5.2 (r252:60911, Jan 24 2010, 17:44:40) [GCC 4.3.2]   remote: ** Mercurial Distributed SCM (version 1.6.2)   remote: ** Extensions loaded:

    The .DS_Store file is a file automatically created by Mac OS X used to index directories.

  • Impact: Currently won’t allow any of my recent check ins to be pushed to the server.
• • Admin: Hugo
  • Date: 1/12/2011
  • Actions: Made two trace files available for download to monotone developer, strace.out and ltrace.out.
  • Reason: Reporting monotone bugs to monotone developers.
  • Impact: More files available in the public download area.
• • Admin: Mando
  • Date: 1/12/2011
  • Actions: Turned on the DISTCLEAN option in neurospaces_cron.
  • Reason: Some machines do not rebuild config.status and attempt to use the old version of the file to create Makefiles. If it attempts to rebuild a Makefile that has since been removed from the workspace it will kill the cronjob. The distclean option deletes config.status and forces it to be rebuilt on every cron run.
  • Impact: Should prevent the cronjob from breaking should a Makefile be deleted from, or moved within a package workspace.
• • Admin: Mando
  • Date: 1/10/2011
  • Actions: Added a blank sspy repository.
  • Reason: Development on the python scheduler is proceeding so a repository will be needed. When the repository is ready will be a simple matter to turn it on and sync.
  • Impact: None
• • Admin: Mando
  • Date: 1/7/2011
  • Actions: Removed the neurospaces_project directory and did a fresh pull.
  • Reason: Due to several file moves done during the python swig bindings the tester could not properly update the workspace, this kills the cron job. Changes from the build machine do not seem to propagate properly to the tester. Also permissions on test scripts were not updating with new cron runs.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 1/5/2010
  • Actions: Checked the tester to see if it installs the built SWIG python libraries correctly.
  • Reason: Simple import test for installed modules seems to be failing. Tester seems to be installing the module correctly, but the test currently fails.
  • Impact: No change.
• • Admin: Hugo
  • Date: 26/12/2010
  • Actions: Continued the work of the previous item after help with a work around from the monotone mailing list. Both neurospaces_pull and neurospaces_update were succesful.
  • Reason: Adding rtxi to repo-genesis3.cbi.utsa.edu
  • Impact: Correctly initialized repository and workspace for the rtxi component on repo-genesis3.cbi.utsa.edu.
• • Admin: Hugo
  • Date: 25/12/2010
  • Actions: Trying to initialize the rtxi repository and pull its content from my laptop in belgium. Directories created, repository initialized. Pull was stopped by a presumed bug in monotone, reported on the monotone list. Will try again after feedback from the list.
  • Reason: Adding rtxi to repo-genesis3.cbi.utsa.edu
  • Impact: None
• • Admin: Hugo
  • Date: 12/12/2010
  • Actions: Removed untarred contents of cpan module installation directories.

    sudo rm -fr Clone-0.31/ Data-Utilities-0.04/ Digest-SHA-5.48/ File-Find-Rule-0.32/ HTML-Table-2.08a/ HTML-Template-2.9/ Inline-0.46/ Number-Compare-0.01/ Test-Simple-0.96/ Text-Glob-0.08/

  • Reason: The tester was still complaining about the permissions of:

    drwx------ 4 root     root       4096 2010-12-11 12:34 HTML-Table-2.08a

  • Impact: Directories removed, hopefully the additional code added to the developer package to remove cpan source code directories after their installation keeps the tester in a sane state from now on.
• • Admin: Mando
  • Date: 12/11/2010
  • Actions: Checked status on cpan directory and its unzipped contents.
  • Reason: The unzipped directories are getting unzipped as root as opposed to random uid/gids as before:

    -rw-r--r-- 1 g3tester g3tester   3780 2010-10-22 12:01 01-Text-Glob-0.08.tar.gz   -rw-r--r-- 1 g3tester g3tester   2123 2010-10-22 12:01 03-Number-Compare-0.01.tar.gz   -rw-r--r-- 1 g3tester g3tester  94716 2010-10-22 12:01 05-Inline-0.46.tar.gz   -rw-r--r-- 1 g3tester g3tester  44744 2010-10-22 12:01 07-Inline-Python-0.37.tar.gz   -rw-r--r-- 1 g3tester g3tester  40002 2010-10-22 12:01 09-Digest-SHA-5.48.tar.gz   -rw-r--r-- 1 g3tester g3tester 105888 2010-10-22 12:01 10-Test-Simple-0.96.tar.gz   -rw-r--r-- 1 g3tester g3tester  15534 2010-10-22 12:01 11-File-Find-Rule-0.32.tar.gz   -rw-r--r-- 1 g3tester g3tester  27832 2010-10-22 12:01 13-Data-Utilities-0.04.tar.gz   -rw-r--r-- 1 g3tester g3tester  11753 2010-10-22 12:01 15-Clone-0.31.tar.gz   -rw-r--r-- 1 g3tester g3tester  23934 2010-12-09 18:10 20-HTML-Table-2.08a.tar.gz   -rw-r--r-- 1 g3tester g3tester  82558 2010-12-09 18:10 25-HTML-Template-2.9.tar.gz   drwxr-xr-x 4 root     root       4096 2010-12-11 12:34 Clone-0.31   -rwxr-xr-x 1 g3tester g3tester    904 2010-12-10 18:10 cpan_install   drwxr-xr-x 5 root     root       4096 2010-12-11 12:34 Data-Utilities-0.04   drwxr-xr-x 7 root     root       4096 2010-12-11 12:34 Digest-SHA-5.48   drwxr-xr-x 6 root     root       4096 2010-12-11 12:34 File-Find-Rule-0.32   drwx------ 4 root     root       4096 2010-12-11 12:34 HTML-Table-2.08a   drwxr-xr-x 6 root     root       4096 2010-12-11 12:34 HTML-Template-2.9   drwxr-xr-x 9 root     root       4096 2010-12-11 12:34 Inline-0.46   drwxr-xr-x 4 root     root       4096 2010-12-11 12:34 Number-Compare-0.01   drwxr-xr-x 6 root     root       4096 2010-12-11 12:34 Test-Simple-0.96   drwxr-xr-x 5 root     root       4096 2010-12-11 12:34 Text-Glob-0.08

  • Impact: None
• • Admin: Mando
  • Date: 12/10/2010
  • Actions: Deleted some directories in the developer package generated from unzipping cpan tarballs.
  • Reason: The unzipped directories were getting corrupted user and group permissions as noted:

    -rw-r--r-- 1 g3tester g3tester   3780 2010-10-22 12:01 01-Text-Glob-0.08.tar.gz   -rw-r--r-- 1 g3tester g3tester   2123 2010-10-22 12:01 03-Number-Compare-0.01.tar.gz   -rw-r--r-- 1 g3tester g3tester  94716 2010-10-22 12:01 05-Inline-0.46.tar.gz   -rw-r--r-- 1 g3tester g3tester  44744 2010-10-22 12:01 07-Inline-Python-0.37.tar.gz   -rw-r--r-- 1 g3tester g3tester  40002 2010-10-22 12:01 09-Digest-SHA-5.48.tar.gz   -rw-r--r-- 1 g3tester g3tester 105888 2010-10-22 12:01 10-Test-Simple-0.96.tar.gz   -rw-r--r-- 1 g3tester g3tester  15534 2010-10-22 12:01 11-File-Find-Rule-0.32.tar.gz   -rw-r--r-- 1 g3tester g3tester  27832 2010-10-22 12:01 13-Data-Utilities-0.04.tar.gz   -rw-r--r-- 1 g3tester g3tester  11753 2010-10-22 12:01 15-Clone-0.31.tar.gz   -rw-r--r-- 1 g3tester g3tester  23934 2010-12-09 18:10 20-HTML-Table-2.08a.tar.gz   -rw-r--r-- 1 g3tester g3tester  82558 2010-12-09 18:10 25-HTML-Template-2.9.tar.gz   drwxr-xr-x 4      502 root       4096 2010-12-10 12:34 Clone-0.31   -rwxr-xr-x 1 g3tester g3tester    866 2010-10-24 08:01 cpan_install   drwxr-xr-x 5      502      502   4096 2010-12-10 12:34 Data-Utilities-0.04   drwxr-xr-x 7 cbiadmin cbiadmin   4096 2010-12-10 12:34 Digest-SHA-5.48   drwxr-xr-x 6      501 staff      4096 2010-12-10 12:34 File-Find-Rule-0.32   drwx------ 4    11003 root       4096 2010-12-10 12:34 HTML-Table-2.08a   drwxr-xr-x 6      500      500   4096 2010-12-10 12:34 HTML-Template-2.9   drwxr-xr-x 9      500      544   4096 2010-12-10 12:34 Inline-0.46   drwxr-xr-x 4 cbiadmin cbiadmin   4096 2010-12-10 12:34 Number-Compare-0.01   drwxr-xr-x 6      501      501   4096 2010-12-10 12:34 Test-Simple-0.96   drwxrwxr-x 5 cbiadmin cbiadmin   4096 2010-12-10 12:34 Text-Glob-0.08

    Permissions seem to be set randomly, not sure of the exact cause.

  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 12/10/2010
  • Actions: Checked machine to diagnose problem with the build of debian and rpm packages.
  • Reason: Without warning the build for rpm and debian packages has failed. No error report in the email.
  • Impact: deb and rpm packages are currently offline.
• • Admin: Mando
  • Date: 12/01/2010
  • Actions: Added Dave’s monotone identifier to the .monotone/write-permissions file of the monotone user running the mtn server.
  • Reason: Identifier must be put into the write-permissions file to allow a user write access even if the key has been added to the database.
  • Impact: Dave can sync to the documentation system.
• • Admin: Mando
  • Date: 11/22/2010
  • Actions: Added Dave’s monotone key to the userdocs system.
  • Reason: Dave needs his key added so that he can add documents.
  • Impact: None.
• • Admin: Hugo
  • Date: 11/08/2010
  • Actions: Checking file timestamps, diffing expected and seen output.
  • Reason: One test that started failing without any apparant reason. The output test of the gshell seems not to respect the output clock anymore, unknown reason.
  • Impact: None.
• • Admin: Mando
  • Date: 10/28/2010
  • Actions: Fixed some issues with the userdocs installation on the repository.
  • Reason: Build was failing due to some permission errors. Errors with the gtube specification images still remain. Difficult to diagnose due to very vague error message. Forums don’t provide much help.
  • Impact: Cronjob for building userdocs on the repo should proceed as normal.
• • Admin: Mando
  • Date: 10/22/2010
  • Actions: Attempted to fix the issues with images on the repository userdocs installation.
  • Reason: All gtube technical specification images are gone. Making a png of an eps fails with the non descriptive ”Unrecoverable error, exit code 1” message. Also some other images have images cropped off on their sides for no apparent reason.
  • Impact: None, tho the issue remains.
• • Admin: Mando
  • Date: 10/21/2010
  • Actions: Set up the documentation system on a cron job on the repository.
  • Reason: Darwin mangles figures, linking the documentation from the repo is a short term fix.
  • Impact: Debian latex should make it easier to fix the issue with the mangled figures.
• • Admin: Mando
  • Date: 10/10/2010
  • Actions: Uploaded some doxygen and epydoc documentation for the gtube source code.
  • Reason: Doesn’t automatically update and there is no hook for epydoc yet. Want to see the look and functionality of both sets of documentation on the server.
  • Impact: Two sets of ”identical” documentation are now on the server, each with their own sets of features and appearances.
• • Admin: Hugo
  • Date: 9/26/2010
  • Actions: Added a line ’LOGDIR: /var/www/html/tests/neurospaces_cron_logs’ to ’/home/g3tester/neurospaces_cron_all.yml’.
  • Reason: Sometimes the tester output of two consecutive cron jobs gets concatenated into one neurospaces_cron.stdout file.
  • Impact: No doubled output anymore for two consecutively executed cron jobs.
• • Admin: Mando
  • Date: 9/16/2010
  • Actions: Inspected permissions on directories served over the internet.
  • Reason: Suspected some files or directories may be owned by root instead of g3tester. Was not the case, permissions were ok.
  • Impact: None
• • Admin: Mando
  • Date: 9/16/2010
  • Actions: Retrieve a copy of the neurospaces_cron configuration file for testing.
  • Reason: Going to retest the output of source tarballs and packages with the same configuration to check for any bugs.
  • Impact: None
• • Admin: Hugo
  • Date: 8/29/2010
  • Actions: Inspecting crontab and tester configuration files.
  • Reason: Sometimes the tester output of two consecutive cron jobs gets concatenated into one neurospaces_cron.stdout file. This seems to happen only when the configuration file “/home/g3tester/neurospaces_cron_all.yml” is used. For those runs, errors are reported twice because they occur twice in the report that reports about two consecutive runs. The line

    LOGDIR: /var/www/html/tests/neurospaces_cron_logs

    is missing from “/home/g3tester/neurospaces_cron_all.yml”, such that neurospaces_cron does not rotate logs in the correct directory (I guess it attempts it either in the /tmp or /root directory). Inside neurospaces_cron the function rotate_logs() is likely also responsible for partial truncation of the output file.

  • Impact: None
• • Admin: Mando
  • Date: 8/22/2010
  • Actions: Removed source code directories building up during packaging.
  • Reason: The unzipped directories were not being cleaned by the packaging process. Extra directories of code were appearing in the doxygen documentation.
  • Impact: Doxygen documentation should no longer have multiple directories of the same code listed.
• • Admin: Hugo
  • Date: 8/22/2010
  • Actions: None
  • Reason: It takes a very long time to execute userdocs-sync and neurospaces_sync. Checking if ssh connection to repo-genesis3 and other servers takes as long (and it does).
  • Impact: None.
• • Admin: Mando
  • Date: 8/20/2010
  • Actions: Added monotone keys for my other machines: the 64 bit desktop, desktop and laptop Virtual machines.
  • Reason: Could not sync to the experiment repository from my other machines.
  • Impact: Public keys for three other machines are now on the experiment repository.
• • Admin: Hugo
  • Date: 8/15/2010
  • Actions: Inspecting directories inside /home/g3tester/neurospaces_project/gshell/source/snapshots/0/
  • Reason: The gshell ns-sli tests are failing, diagnosis only.
  • Impact: None.
• • Admin: Mando
  • Date: 8/13/2010
  • Actions: Deleted a directory with outdated gtube doxygen docs.
  • Reason: It is not fully integrated into the cronjob yet so had to be done manually.
  • Impact: N/A
• • Admin: Mando
  • Date: 8/13/2010
  • Actions: Installed fakeroot.
  • Reason: fakeroot was not installed and it used by the package builder. Updated debian server document as needed.
  • Impact: N/A
• • Admin: Hugo
  • Date: 8/13/2010
  • Actions: Inspecting directories inside /home/g3tester/neurospaces_project/gshell/source/snapshots/0/
  • Reason: The gshell ns-sli tests are failing, diagnosis only. It looks as if the test scripts do not produce output files because the output directories do not exist.

    *** Application output file: ./..//output/cell.out (in /home/g3tester/neurospaces_project/gshell/source/snapshots/0/tests)   cat: ./..//output/cell.out: No such file or directory   *** Expected output file: /usr/local/ns-sli/tests/specifications/strings/traub91_asym.ssp

    and

    *** Error: 3:Child PID 8006 exited with status 11 (Do we see the simulation time after the simulation has finished ?, package gshell, sli.t, error_count 2)   ...   *** Application output file: ./..//tests/results/PurkM9_soma_1.5nA (in /home/g3tester/neurospaces_project/gshell/source/snapshots/0/tests)   cat: ./..//tests/results/PurkM9_soma_1.5nA: No such file or directory   *** Expected output file: /usr/local/ns-sli/tests/specifications/strings/PurkM9_soma_1.5nA.g3-double

  • Impact: None.
• • Admin: Hugo
  • Date: 8/12/2010
  • Actions: Attempt to install XML::Simple from CPAN. Got Expat.xs:12:19: error: expat.h: No such file or directory, so also installed libexpat1-dev
  • Reason: The exchange tests were failing.
  • Impact: Installed libexpat1-dev and XML::Simple and its CPAN dependencies. perl -e ’use XML::Simple’ is successful.
• • Admin: Mando
  • Date: 8/12/2010
  • Actions:
    • 1. Restarted the monotone repository servers.
    • 2. Installed package building software.
  • Reason:
    • 1. At some point late 8/11 or early 8/12 the monotone repositories became unconnectable. Cause unknown for now but will continue to monitor the situation.
    • 2. The noon cron run uses dpkg and rpmbuild to build packages, needed to install them. Updated neurospaces-cron document to reflect this.
• • Admin: Mando
  • Date: 8/11/2010
  • Actions: Continued installation of server software.
  • Reason:
    • Needed more packages installed for doxygen, source-highlight, tex, etc.
    • Added and documented how to install a package to set the clock time.
    • Cronjob for Noon did not go off so testing crontab with a wall command:

      * * * * * root wall /etc/testmsg

      Where testmsg is a text file with a broadcast message.

  • Impact:
    • Doxygen documentation should be updated with graphs and regression tests should be updated on cronrun.
    • Time is properly set on the server.
    • cron daemon was verified to work correctly.
• • Admin: Hugo
  • Date: 8/10/2010
  • Actions: Upgrade the monotone repository to version 0.48 (mtn db migrate). Manual upgrade of the developer package (neurospaces_pull, neurospaces_update, neurospaces_install on the developer package only). Then as the monotone user: neurospaces_create_directories and neurospaces_init –regex experiment.

    as monotone: cat monotone_keys/mando.pubkey monotone_keys/hugo.pubkey monotone_keys/allan.pubkey | mtn --db=~/neurospaces_project/MTN/experiment.mtn read   as root: sudo -H -u monotone nohup neurospaces_serve &   on my laptop: neurospaces_sync

  • Reason: Developer package configuration was updated with the experiment software package. The experiment monotone server must be created and started manually.
  • Impact: All repositories (includes the experiment repository) should be up and running again.
• • Admin: Mando
  • Date: 8/10/2010
  • Actions: Started installation of repository software.
  • Reason: Repository machine was reinstalled.
  • Impact: N/A
• • Admin: Hugo
  • Date: 8/10/2010
  • Actions: Check if the repo was already reinstalled.
  • Reason: Check if the repo was already reinstalled.
  • Impact: None.
• • Admin: Mando
  • Date: 8/9/2010
  • Actions: Restarted monotone repositories.
  • Reason: Userdocs repository got ”stuck”
  • Impact: N/A
• • Admin: Mando
  • Date: 8/9/2010
  • Actions: Backed up system and data files.
  • Reason: Need the files to set up the server after reinstallation.
  • Impact: N/A
• • Admin: Mando
  • Date: 7/9/2010
  • Actions: Started monotone server, removed old 32-bit RPM files and directories to upload newer versions, and did an integrity check on the server.
  • Reason: Server had a new hard drive installed and rebuilt the RAID so needed to start the monotone server that does not start on boot and make sure it was running properly.
  • Impact: N/A
• • Admin: Mando
  • Date: 7/6/2010
  • Actions: Removed old Mac OSX NS-SLI package from the downloadable area.
  • Reason: Package has a known bug in determining the execution path.
  • Impact: Should not affect Repository or Tester operations.
• • Admin: Mando
  • Date: 7/2/2010
  • Actions: Removed simrc file in g3tester home directory after fixing the problem with finding the simrc in the executable directory.
  • Reason: The .simrc-ns-sli file in the home directory was masking a bug in determining the simrc file in the executable directory.
  • Impact: Removing the file will allow the tests to continue with a file that is under monotone control.

darwin.cbi.utsa.edu

This machine hosts the GENESIS website as well as the output from the user documentation system. All of the user documentation building tasks are done through a single non-root user account named ǵenesis

• • Admin: Mando
  • Date: 4/5/2012
  • Actions: Checked the install path for perl using the Config module.
  • Reason: Perl is attempting to install to a path that centOS does not search for perl modules in. It seemed to work previously possibly due to modules that were in the correct path remaining from previous runs.
  • Impact: Website should continue running.
• • Admin: Mando
  • Date: 1/23/2012
  • Actions: Updated an .inc php file in drupal to use preg_match instead of ereg.
  • Reason: ereg was causing a warning due to deprecation in php 5+
  • Impact: Website should cease showing warnings on pages.
• • Admin: Hugo
  • Date: 3/12/2011
  • Actions: Installed the openoffice.org-xsltfilter.x86_64 and openoffice.org-writer.x86 packages.
  • Reason: The jodconverter utility was failing with a ’conversion failed: could not load input document’ error.
  • Impact: The documentation system should now incorporate word files into the website, waiting for next run.
• • Admin: Hugo
  • Date: 3/8/2011
  • Actions: Installed jodconverter for root, Mando will move things to appropriate locations such that jodconverter is available for all users.
  • Reason: The jodconverter utility configures openoffice to convert word files to other formats and is now used by the documentation system to convert word files to html and pdf.
  • Impact: The documentation system should now incorporate word files into the website.
• • Admin: Hugo
  • Date: 9/21/2010
  • Actions: Comparing monotone version control identifiers, checking output of latex compilation. The cbi-archtecture document is tagged as draft, so it is not compiled by the documentation system during a build. This behavior is undesirable for some documents but is ok for others. Maybe we need to introduce a new tag for this type of situation. Added to the developer TODOs.
  • Reason: The cbi-archtecture document fails to compile, but the errors are not reported by the documentation system.
  • Impact: None.
• • Admin: Hugo
  • Date: 9/21/2010
  • Actions: Check for the availability of variants of the script rst2pdf.
  • Reason: The rst2pdf script is not found (although rst2html is).
  • Impact: None.
• • Admin: Hugo
  • Date: 9/21/2010
  • Actions: Check if the document input-models-notes was generated on darwin. It looks as if the missing rst2pdf script makes for a situation where the successfully build html is not published.
  • Reason: input-models-notes is not available from the website index page.
  • Impact: None.
• • Admin: Mando
  • Date: 9/20/2010
  • Actions: Installed unrtf and docutils from source (Performed as root).
  • Reason: The rst2html and unrtf commands are not found and not on yum.
  • Impact: Documentation features should be enabled.
• • Admin: Hugo
  • Date: 9/20/2010
  • Actions: Checking for the existence of the commands rst2html and unrtf.
  • Reason: The rst2html and unrtf commands are required for document building.
  • Impact: None
• • Admin: Hugo
  • Date: 8/31/2010
  • Reason: Checking the availability of restructured text on darwin.
  • Actions: Unsuccessful attempts to install with ’yum install python-docutils’.
  • Impact: None expected.
• • Admin: Mando
  • Date: 8/23/2010
  • Actions: Installed the rest of the dependencies needed to get all of GENESIS 3 working as listed in . Should be noted that all installation was done via root, not the genesis user.
  • Reason: Userdocs needs the gshell functional to dynamically create documentation snippets.
  • Impact: All tests (sans the ones with known issues) are now functional on darwin.
• • Admin: Mando
  • Date: 7/12/2010
  • Reason: Need to update website to search through userdocs.
  • Actions: Made some changes to the website software that was to allow searching through all imported html files with the intent that it would allow quick searches through userdocs.
  • Impact: Did not go as planned, the imported static html pages do not seem to be properly indexed in the site search. Needed to update the cron script to the newest version due to a new feature needed, this should not affect operations.
• • Admin: Mando
  • Date: 7/10/2010
  • Reason: Need to update website to search through userdocs.
  • Actions: Did work on the website software.
  • Impact: N/A
• • Admin: Mando
  • Date: 7/7/2010
  • Reason: Need to add a sitemap to the userdocs.
  • Actions: Did work on the website software.
  • Impact: N/A
• • Admin: Mando
  • Date: 7/6/2010
  • Reason: Need to add some files to work with google custom search.
  • Actions: Did work on the website software.
  • Impact: N/A
• • Admin: Mando
  • Date: 7/2/2010
  • Reason: Website software and database needed updating.
  • Actions: Did work on the website software.
  • Impact: N/A

Mac Tester

The Mac tester is an iMac machine located at CBI that runs the tester on a cronjob. It is also used for debugging and building packages for 32-bit Mac OSX Snow Leopard.

System specs:

• Processor: 2 Ghz Intel Core Duo
• Memory: 2 GB 667 Mhz DDR2 SDRAM
• Operating System: Mac OSX Snow Leopard (Version 10.6.4)
• Monotone version: monotone 0.99.1 (base revision: 8973482283db7c36780dce2b54721ccc0f5b7388)

• • Admin: Mando
  • Date: 3/19/2012
  • Actions: Rebooted the mac tester.
  • Reason: A number of sleeping processes had accumulated and caused the ’fork’ function to fail, thus making it incapable of running the cron script.
  • Impact: Tester should proceed as normal. Will monitor for more stray perl processes.
• • Admin: Mando
  • Date: 5/2/2011
  • Actions: Performed a neurospaces_revert and a neurospaces_update.
  • Reason: Needed to ”unstick” the tester after it was caught in an unusable state like the regular tester.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 4/19/2011
  • Actions: Added a cronjob that builds and tests universal binaries.
  • Reason: Recently added the functionality for building multiple architectures so it should be tested at least once a day.
  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 1/28/2011
  • Actions: Activated the sspy repository for testing. Installed PyYaml.
  • Reason: Need to start syncing code between machines. Differences between python on mac and linux creates a need for testing on the Mac.
  • Impact: Tester should proceed as normal. Will monitor for any workspace issues.
• • Admin: Mando
  • Date: 1/20/2011
  • Actions: Created a copy of the existing workspaces and removed the ’glue/swig/python_old/heccer’ directory that was present and performed a revert to update to what is currently in the repository.
  • Reason: Apparently there is a bug in monotone that does not resolve path changes involving only the changing of a letters case.

    Mac-Tester:0 g3tester$ mtn update   mtn: updating along branch ’0’   mtn: selected update target 2898afd0a666f80d05f479d2f9d9d65d50e7c326   mtn: [left]  1f09160cf4ba710e1afbc64b56d8cd2702dc6bc3   mtn: [right] 2898afd0a666f80d05f479d2f9d9d65d50e7c326   mtn: misuse: workspace is locked   mtn: misuse: you must clean up and remove the \_MTN/detached directory

    The error stems from a problem in monotone on Mac OS X that ignores case. The updating of the workspace initially produces this error, and then generated the _MTN/detached directory that locks the workspace, changing the error message.

    mtn: updating along branch ’0’   mtn: selected update target 2898afd0a666f80d05f479d2f9d9d65d50e7c326   mtn: [left]  1f09160cf4ba710e1afbc64b56d8cd2702dc6bc3   mtn: [right] 2898afd0a666f80d05f479d2f9d9d65d50e7c326   mtn: adding glue/swig/python_old/Heccer   mtn: misuse: rename target ’glue/swig/python\_old/Heccer’ already exists

    Sent an email to the monotone mailing list regarding this error. Note I have emailed the monotone mailing list about something similar in the past; when monotone has had conflicts updating workspaces when there is a file and a directory sharing the same name.

  • Impact: Should proceed as normal.
• • Admin: Mando
  • Date: 1/14/2011
  • Actions:
    • Admin: Mando
    • Date: 7/22/2011
    • Actions: Reinstalled the developer package from a tarball.
    • Reason: Seemed the tester got stuck in the time period when the script ’neurospaces_repositories’ was missing. As a result neurospaces_build would not execute.
    • Impact: Should proceed as normal
    • 1. Created a copy of the workspace to examine the contents for potential bugs.
    • 2. Updated version of monotone from 0.47 to 0.99.1 to see if this alleviates some of the issues before posting to the monotone mailing list.
  • Reason: The mac tester is refusing to pull or update to any revisions past those checked in on 1/04/2011. This leaves it in a state where it cannot properly update the workspace and leaves some bugs that were since fixed:

    Mac-Tester:0 g3tester$ mtn log --last 2   o   ----------------------------------------------------------------------   |   Revision: da53dc5ab63f91184f25068613e4d39eadc66c2e   |   Parent:   1cd5e99c7d82b732a41431424d45b81db7d75f00   |   Author:   mandorodriguez+Ubuntu.VM.Home.i686@gmail.com   |   Date:     01/04/2011 23:45:14   |   Branch:   0   |   |   Changelog:   |   |   1. Fixed a permission error on the python import test.   |   |   2. Fixed the module import in the passive compartment test.   |   |   Changes against parent 1cd5e99c7d82b732a41431424d45b81db7d75f00   |   |     patched  tests/python/c1c2p1.py   |     patched  tests/python/import_test.py   o   ----------------------------------------------------------------------   |   Revision: 1cd5e99c7d82b732a41431424d45b81db7d75f00   |   Parent:   e592223f033e50e8da3c55d4d8182ea3f776161b   |   Author:   mandorodriguez@gmail.com   |   Date:     01/04/2011 16:29:28   |   Branch:   0   |

    Most recent log entry at the time of the last cron job should be on 01/13/2011:

    o   -----------------------------------------------------------------   |   Revision: 278c4ff3486df30271454f28f13740bb7ed21063   |   Ancestor: 298e74b52e74262741c96da833e4640cb266d9bb   |   Author: mandorodriguez@gmail.com   |   Date: 01/13/2011 17:42:49   |   Branch: 0   |   |   Modified files:   |           glue/swig/python/heccer/__init__.py   |           glue/swig/python/heccer/errors.py   |   |   ChangeLog:   |   |   1. Added an exception for not resolving an address in the python Swig bindings.

    No other machine seems to have this issue. Attempted many fixes but the end result was always the heccer repository failing to pull down any revision past da53dc5ab63f91184f25068613e4d39eadc66c2e on 1/4/2011. The only way to prevent the error from occurring again was to perform a repull the heccer repository and creating a new workspace. After doing a new pull, all revisions were present. The bug is reproducible with saved copies of the repository so old repositories in monotone version 0.47, migrated versions of the faulty repos in monotone 99.1 have been preserved.

  • Impact: Tester should proceed as normal.
• • Admin: Mando
  • Date: 4/29/2011
  • Actions: Removed the directory ”/usr/local/glue/swig/python/Neurospaces”
  • Reason: Python module convention says that modules and module directories should all be in lowercase. Installer does not install the ’neurospaces’ directory when ’Neurospaces’ is present. Automake clean rules do not delete directories unless you issue an explicit rm command, so it had to be removed manually.
  • Impact: Should proceed as normal.
  • Admin: Mando
  • Date: 1/13/2011
  • Actions: Got the mac tester back online.
  • Reason: It was previously switched off to be moved to a different desk.
  • Impact: Should proceed as normal.
• • Admin: Mando
  • Date: 9/14/2010
  • Reason: An error citing python as being not executable.
  • Actions: env now finds /usr/bin before the standard mac python location. Added the standard mac location to the crontab path as it cites only the regular unix locations.
  • Impact: Test should not output an error any longer.
• • Admin: Mando
  • Date: 9/9/2010
  • Reason: Observed build errors (die messages) in the tester email.
  • Actions: Inspected project directories and performed a distclean and uninstall.
  • Impact: None
• • Admin: Mando
  • Date: 9/3/2010
  • Reason: Chronic test failures (which have been verified to work correctly).
  • Actions: Examined tester expected and seen files since they can’t be available over the internet.
  • Impact: As of the time of logout, None.
• • Admin: Mando
  • Date: 8/19/2010
  • Reason: Build is failing.
  • Actions: Performed an uninstall and distclean.
  • Impact: Build is functioning as normal.
• • Admin: Mando
  • Date: 8/19/2010
  • Reason: Build is failing.
  • Actions: Logged on to diagnose problem.
  • Impact: N/A
• • Admin: Mando
  • Date: 7/14/2010
  • Reason: Prevent tester from performing unnecessary merging.
  • Actions: Turn on the o automated mergeseature in the developer package.
  • Impact: N/A
• • Admin: Mando
  • Date: 7/12/2010
  • Reason: Tester is ”stuck” due to a bug in monotone that causes a merge conflict.
  • Actions: Made a copy of the ”stuck” monotone repositories and did a fresh pull. Since monotone version is 0.47, will check to see if this is a new bug or one that was previously fixed.
  • Impact: Tester now proceeds as normal.
• • Admin: Mando
  • Date: 7/6/2010
  • Reason: Allow more developers to see test runs on the mac.
  • Actions: Added Hugo to the Mac Tester email.
  • Impact: N/A