Differences

This shows you the differences between two versions of the page.


psi:general_notes [2020/06/10 21:42] (current) – created - external edit 127.0.0.1
Line 1: Line 1:
 +~~ODT~~
 +====== Notes for PSI ======
 +===== Show free disk space =====
 +  df -h /media/disk/
 +===== Rsync the files to HD =====
 +synchronisierung auf workstation ganz links
 +  * grosse disk einstecken
 +  * evtl. mit `sudo gparted` formatieren, damit wirklich alles leer ist...
 +  * evtl. mit `dmesg` schauen, was lso ist, wenn nix passiert
 +  *`cd ~`, damit im root-verzeichnis\
 +  * im terminal eingeben:
  
 +  rsyncdir -t 60000 --no-g --exclude-from=exclude_file.txt /sls/X02DA/data/e11126/Data10/BeamTime /media/MedionHDDrive2GO/
 +
 +  `-t 60000`
 +erhoeht die abbruchzeit, damit synchronisation nicht gestoppt wird, falls mal laenger keine daten auf Data10 geschrieben werden.
 +<note>--no-g konserviert die Gruppenzugehoerigkeit, und ist seit Beamtime 2010e evtl. noetig</note>
 +  `--exclude-from=exclude_file.txt`
 +filelist, die die zu exkludierenden files enthaelt. in unserem fall meist
 +  sin
 +  viewrec
 +  tif
 +  tif_ang
 +  cpr
 +  fcr
 +  rec_DMP
 +falls die rohdaten erwuenscht sind, dann `tif` aus dem exclude_file entfernen. `tif_ang` sind die resortierten 360grad-projektionen, die chris' skript mit hardlinks generiert. 'cpr' und 'fcr' kommen von der holotomography mir rajmund, rec_DMP entstehen, wenn mann falsche parameter bei der rekonstruktion eingibt.
 +  /sls/X02DA/data/e11126/Data10/2009a
 +Verzeichnis mit den Messdaten auf File-Server (einfach herauszufinden mit `pwd`)
 +  /media/disk/
 +Root-Verzeichnis der Daten auf der HD. Ohne z.B. 2009a, denn rsyncdir macht sonst `/media/disk/2009a/2009a/blabla`
 +
 +am ende disk mit 
 +  eject /media/FestPlattenName
 +unmounten.
 +
 +===== backup to archivftp.psi.ch =====
 +Start a Terminal and logon to the 6<sup>th</sup> node of the Cluster:
 +  ssh e11126@x02da-cn-6
 +
 +Change to the directory you need to send to the Backupserver with (for example)
 +  cd Data3/2010a
 +or for example
 +  cd Data10/2009f
 +
 +Then you can generate a Textfile containing a List with Files to backup using the commands:
 +  ls -d */tif > backupfile.txt
 +  ls -d */rec* >> backupfile.txt
 +  ls -d mrg/*/rec* >> backupfile.txt
 +  ls -d mrg/*/tif* >> backupfile.txt #only if you want to backup the merged tiffs!
 +
 +  * 'ls -d */tif' lists all the subdirectories containing the Projections in the directory you're in and '> backupfile.txt' writes the output of this command to a textfile with the chosen name.
 +  * The following commands look for other directories ('ls -d */rec*' looks for all instances of rec*-Directories, e.g. rec_8bit or rec_16bit) and appends them (using '>>') to the same file. The appending is important, or else you'll loose what you've entered in the previous lines!
 +  * In the end we're also looking for Projections and Reconstructions in the 'mrg' subfolder, where we've written the merged Files using the [[widefieldscan]]-workflow.
 +
 +To actually start the backup, you need to issue the following command (in the directory where you generated the backupfile):
 +  bl-ftp2arch.sh -u e11126 -e david.haberthuer@psi.ch -b -P -f backupfile.txt
 +  
 +This starts the archiver-script for the Schittny-eAccount (-u), notifies David about success/failure (-e, email goes only to @psi-Addresses), performs the archival in the background (-b), asks for the archivFTP-Password (-P) and uses backupfile.txt as list of files to backup (-f).
 +
 +If you want to monitor the progress, it's probably best if you run the command without the -b flag inside a [[https://www.google.ch/search?q=linux+screen|screen]]-session, so you can hangup the terminal and come back.
 +
 +===== Restore backup from archivftp.psi.ch =====
 +Either follow [[http://sls.web.psi.ch/view.php/beamlines/ms/xtm/manuals/index.html#restore|these steps]] on the TOMCAT-website or:
 +
 +  * Connect to slslc05.psi.ch per SSH
 +  * Start gFTP to have a graphical interface to the FTP-server. Start it with issuing "gftp" in the terminal.
 +  * login as either e11126@archivftp.psi.ch or e10277@archivftp.psi.ch. The passwords are known by the Beamline-Crew or me. 
 +  * Browse to the directory which you want to restore.
 +  * Try to transfer the data. Since archivftp.psi.ch stores everything on tape, you'll probably get an error the first time. Just retry after some minutes, then everything should work like a charm.
 +
 +===== DicoClient =====
 +Falls jobs manuell submitted wurden und nicht mit dem RecoManager angeschaut werden koennen, dann kann auf x02da-cons-2
 +  cd /usr/local/cluster/DiCoClient/
 +  java -jar DiCoClient.jar
 +ausgefuehrt werden. Dies started den DicoClient ([[https://controls.web.psi.ch/cgi-bin/twiki/view/Main/DiCat|Distibuted Computing Client]]). Mit "lj ex a" sieht mensch alle momentan ausgefuehrten, der Rest wird mit "h" erklaert.
 +
 +===== rename multiple files =====
 +either
 +  rename R243b10_ R243b10_mrg R243b10_???.tif
 +or use workaround like below if "rename" does not work with more than 1000 files (seems to be the case over ssh/x-server...)
 +  rename R243b10_0 R243b10_mrg0 R243b10_0???.tif
 +  rename R243b10_1 R243b10_mrg1 R243b10_1???.tif
 +  rename R243b10_2 R243b10_mrg2 R243b10_2???.tif
 +  rename R243b10_3 R243b10_mrg3 R243b10_3???.tif
 +  rename R243b10_4 R243b10_mrg4 R243b10_4???.tif
 +  ...
 +
 +===== Camserver restart =====
 +  rdesktop pc6189 -g 1250x950 -a 16
 +oder mit
 +  alias | grep CCD
 +rausfinden, welcher PC momentan grad zustaendig ist...
 +===== Reset Camera =====
 +  * Start CamWare and take any picture (snap shot or movie)
 +  * Close CamWare
 +  * Start Image-Pro Plus
 +  * Start Camera Server FIFO
 +===== Image-Pro Plus =====
 +  * Camera control -> Exposure, binning, etc.
 +  * Histogram (Histogram of image, directly linked to a particular image)
 +  * Flat: 40'000 - 50'000 counts.
 +  * Arithmetic and logical operations -> divide image by flat -> min. 2000 - 3000 counts in new images. If not -> higher energy!!
 +  * Enhance - Display range: Gives histogram of current image and defines display range. May be use constantly.