Differences

This shows you the differences between two versions of the page.

Link to this comparison view

psi:general_notes [2013/07/04 08:45]
psi:general_notes [2013/07/04 08:45] (current)
Line 1: Line 1:
 +~~ODT~~
 +====== Notes for PSI ======
 +===== Show free disk space =====
 +  df -h /​media/​disk/​
 +===== Rsync the files to HD =====
 +synchronisierung auf workstation ganz links
 +  * grosse disk einstecken
 +  * evtl. mit `sudo gparted` formatieren,​ damit wirklich alles leer ist...
 +  * evtl. mit `dmesg` schauen, was lso ist, wenn nix passiert
 +  *`cd ~`, damit im root-verzeichnis\
 +  * im terminal eingeben:
  
 +  rsyncdir -t 60000 --no-g --exclude-from=exclude_file.txt /​sls/​X02DA/​data/​e11126/​Data10/​BeamTime /​media/​MedionHDDrive2GO/​
 +
 +  `-t 60000`
 +erhoeht die abbruchzeit,​ damit synchronisation nicht gestoppt wird, falls mal laenger keine daten auf Data10 geschrieben werden.
 +<​note>​--no-g konserviert die Gruppenzugehoerigkeit,​ und ist seit Beamtime 2010e evtl. noetig</​note>​
 +  `--exclude-from=exclude_file.txt`
 +filelist, die die zu exkludierenden files enthaelt. in unserem fall meist
 +  sin
 +  viewrec
 +  tif
 +  tif_ang
 +  cpr
 +  fcr
 +  rec_DMP
 +falls die rohdaten erwuenscht sind, dann `tif` aus dem exclude_file entfernen. `tif_ang` sind die resortierten 360grad-projektionen,​ die chris' skript mit hardlinks generiert. '​cpr'​ und '​fcr'​ kommen von der holotomography mir rajmund, rec_DMP entstehen, wenn mann falsche parameter bei der rekonstruktion eingibt.
 +  /​sls/​X02DA/​data/​e11126/​Data10/​2009a
 +Verzeichnis mit den Messdaten auf File-Server (einfach herauszufinden mit `pwd`)
 +  /​media/​disk/​
 +Root-Verzeichnis der Daten auf der HD. Ohne z.B. 2009a, denn rsyncdir macht sonst `/​media/​disk/​2009a/​2009a/​blabla`
 +
 +am ende disk mit 
 +  eject /​media/​FestPlattenName
 +unmounten.
 +
 +===== backup to archivftp.psi.ch =====
 +Start a Terminal and logon to the 6<​sup>​th</​sup>​ node of the Cluster:
 +  ssh e11126@x02da-cn-6
 +
 +Change to the directory you need to send to the Backupserver with (for example)
 +  cd Data3/2010a
 +or for example
 +  cd Data10/​2009f
 +
 +Then you can generate a Textfile containing a List with Files to backup using the commands:
 +  ls -d */tif > backupfile.txt
 +  ls -d */rec* >> backupfile.txt
 +  ls -d mrg/*/rec* >> backupfile.txt
 +  ls -d mrg/*/tif* >> backupfile.txt #only if you want to backup the merged tiffs!
 +
 +  * 'ls -d */tif' lists all the subdirectories containing the Projections in the directory you're in and '> backupfile.txt'​ writes the output of this command to a textfile with the chosen name.
 +  * The following commands look for other directories ('ls -d */​rec*'​ looks for all instances of rec*-Directories,​ e.g. rec_8bit or rec_16bit) and appends them (using '>>'​) to the same file. The appending is important, or else you'll loose what you've entered in the previous lines!
 +  * In the end we're also looking for Projections and Reconstructions in the '​mrg'​ subfolder, where we've written the merged Files using the [[widefieldscan]]-workflow.
 +
 +To actually start the backup, you need to issue the following command (in the directory where you generated the backupfile):​
 +  bl-ftp2arch.sh -u e11126 -e david.haberthuer@psi.ch -b -P -f backupfile.txt
 +  ​
 +This starts the archiver-script for the Schittny-eAccount (-u), notifies David about success/​failure (-e, email goes only to @psi-Addresses),​ performs the archival in the background (-b), asks for the archivFTP-Password (-P) and uses backupfile.txt as list of files to backup (-f).
 +
 +If you want to monitor the progress, it's probably best if you run the command without the -b flag inside a [[https://​www.google.ch/​search?​q=linux+screen|screen]]-session,​ so you can hangup the terminal and come back.
 +
 +===== Restore backup from archivftp.psi.ch =====
 +Either follow [[http://​sls.web.psi.ch/​view.php/​beamlines/​ms/​xtm/​manuals/​index.html#​restore|these steps]] on the TOMCAT-website or:
 +
 +  * Connect to slslc05.psi.ch per SSH
 +  * Start gFTP to have a graphical interface to the FTP-server. Start it with issuing "​gftp"​ in the terminal.
 +  * login as either e11126@archivftp.psi.ch or e10277@archivftp.psi.ch. The passwords are known by the Beamline-Crew or me. 
 +  * Browse to the directory which you want to restore.
 +  * Try to transfer the data. Since archivftp.psi.ch stores everything on tape, you'll probably get an error the first time. Just retry after some minutes, then everything should work like a charm.
 +
 +===== DicoClient =====
 +Falls jobs manuell submitted wurden und nicht mit dem RecoManager angeschaut werden koennen, dann kann auf x02da-cons-2
 +  cd /​usr/​local/​cluster/​DiCoClient/​
 +  java -jar DiCoClient.jar
 +ausgefuehrt werden. Dies started den DicoClient ([[https://​controls.web.psi.ch/​cgi-bin/​twiki/​view/​Main/​DiCat|Distibuted Computing Client]]). Mit "lj ex a" sieht mensch alle momentan ausgefuehrten,​ der Rest wird mit "​h"​ erklaert.
 +
 +===== rename multiple files =====
 +either
 +  rename R243b10_ R243b10_mrg R243b10_???​.tif
 +or use workaround like below if "​rename"​ does not work with more than 1000 files (seems to be the case over ssh/​x-server...)
 +  rename R243b10_0 R243b10_mrg0 R243b10_0???​.tif
 +  rename R243b10_1 R243b10_mrg1 R243b10_1???​.tif
 +  rename R243b10_2 R243b10_mrg2 R243b10_2???​.tif
 +  rename R243b10_3 R243b10_mrg3 R243b10_3???​.tif
 +  rename R243b10_4 R243b10_mrg4 R243b10_4???​.tif
 +  ...
 +
 +===== Camserver restart =====
 +  rdesktop pc6189 -g 1250x950 -a 16
 +oder mit
 +  alias | grep CCD
 +rausfinden, welcher PC momentan grad zustaendig ist...
 +===== Reset Camera =====
 +  * Start CamWare and take any picture (snap shot or movie)
 +  * Close CamWare
 +  * Start Image-Pro Plus
 +  * Start Camera Server FIFO
 +===== Image-Pro Plus =====
 +  * Camera control -> Exposure, binning, etc.
 +  * Histogram (Histogram of image, directly linked to a particular image)
 +  * Flat: 40'000 - 50'000 counts.
 +  * Arithmetic and logical operations -> divide image by flat -> min. 2000 - 3000 counts in new images. If not -> higher energy!!
 +  * Enhance - Display range: Gives histogram of current image and defines display range. May be use constantly.