EK PythonLib


by Evgenii Kovrigin


Contents


  1. Filename manipulation 
  2. NMR machine monitoring/downloads
  3. Pulse sequence analysis
  4. MolMol data preparation
  5. Misc
  6. Modules used in Python programming
  7. Sparky files preparation/processing
  8. Sparky extensions and related modules
  9. NOTES on new Sparky extension development
  10. Analysis of Sparky exported data 
  11. nmrPipe-directed data processing
  12. ModelFree related 
  13. Calculations of chemical equilibria (for ITC) 
  14. Pymol modules
  15. Relaxation data analysis
  16. Protein sequence analysis





Filename manipulation 

remove_part.py - xxx.aaa.bbb => xxx.bbb
add_prefix.py - aaa.bbb => prefix.aaa.bbb 
replace_part.py :          aaa.xxx.bbb ->  aaa.yyyyyy.bbb
   

 
 

NMR machine monitoring/downloads

email_test.py - testing accessibility of e-mail address for watch_magnet.py

Varian spectrometer tools

watch_magnet3.py - Monitoring the status of the Varian spectrometer (lock, temperature, overflows)
get_exp2.py - Utility to download the experiment from the spectrometer
dof2_calc.py - dof2 calculation on Varian

Bruker spectrometer tools

watch_bruker.py - monitoring progress of acquisition on Bruker instrument (watches incrementation of time stamp on the fid files)

offset_calc.py - General utility for calculation of carrier offsets

water_shift.py - calculate water frequency for the specified temperature
   
   


Pulse sequence analysis


calculate_time.py - analyses the pulse sequence for delays and pulse lengths and
                                presents a summary which enables you to calculate total
                                duration of the sequence
   
   

ek_rccpmg.py - generate lists of tau delays and number of cycles for rcCPMG experiment performed by ek_rccpmg15Ng pulse program. Usually resides with the spectral data from such an experiment.


analyze_interleaved_replicates.py - analyzes the spectral data from Bruker instrument acquired using pseudo 3D mode set up with ek_rccpmg.py to output serial numbers of planes for all specific CPMG frequencies
   


MolMol data preparation

   
all the below: Generation of the colorizing macro for molmol to enable painting of molmol structures
----------------------------------
S2_to_molmol2.py, S2_to_molmol.py
dS2_to_molmol.py
value_to_molmol.py
value_to_molmol_v2.py
   
mm_select_group.py - Generation of molmol command for selection of a group of residues
   

Misc

rt.py             -     measurement of the run-time of the program
   
queuer.py     -     to queue execution of a program in different directories
   
characters.py           -     Determination of color codes of symbols
symbol_codes.py     -     Test letter codes from-to (decimal)
replier.py
xmgr2ps.py      -    the utility to convert all XMGR files in the directory into the PostScript format
topdf.py            -     to convert all eps files in the directory to pdfs
   
 

Modules used in Python programming


mylib.py
colors.py
curvefit3_functions.py
curvefit3.py
molmol.py
mysys.py

Text data processing:

residues.py       -     indexed lists: WordDict, Annn_Text, Annn_Num (most used)
textfile.py         -     work with TEXT files of the ARBITRARY format
keyfile.py         -     work with files indexed in the first column (.ini files)
(keywordfile.py) - (old version - DON'T USE IT)
 
run_parallel.py - parallel processes manipulation and test procedure (SSH works!!!)
   
gnuplot.py - the function to run gnuplot in batch mode
remove_empty_lines.py - the function to prepare data for gnuplot - it chokes on empty lines
   
myPDB.py - a set of functions to deal with PDB files



Sparky files preparation/processing


make_sp_files.py - Generates the sparky .save files for a number of .sp files and the project using
common template file (without MOLECULE/CONDITION set! - see next)

make_save_files.py
- does the same job but uses template file with MOLECULE/CONDITION set
   
unscrew_savefiles.py - make .save and .proj files movable (replaces paths)

add_peak_labels.py - add consequtively numbered labels to ?-? peaks from automatic peak picking
   

import_xeasy.py - a utility to extract assignments from XEASY files and generate Sparky .save file. It also helps import assignment results from GARANT

predict_aa_type.py - an extension to run type_prob program by Grzesiek and Bax to predict amino acid type using carbon chemical shifts. Interactively select Ca and Cb peaks and get output of type_prob into a Python shell.


Sparky extensions and related modules



my_sparky_lib.py

er, zz
- new_extension.py
eC - adjust_contours.py
ec - center_peaks.py    - ec
eO - center_one_peak.py


el - finding a list of peaks with intensities below specified S/N ratio (needs S/N specifically detemined by rm and inserted into st window , because automatic S/N measure is inaccurate) check_low_intensity_peaks.py

copy_assignments_2.py



eo - detection of overlapped residues (assigned ones only) detect_overlap_sparky.py
detection tolerances are set in detect_overlap.ini



es - displaying a series of spectra as heights vs. index display_heights_vs_index.py
eS - (same as above) for VERY MANY spectra (such as large interleaved experiments) display_heights_vs_index.large_spectral_set.py



ef - find_different_assignments.py 
fold_assignments.py
ee - integrate.sp.py - integration of the peak (for subsequent fitting)
rm - regionrmsd.py
eh - relax_exponential_decay.py - 'eh' enhanced version of 'rh'
relax_double_exponential_decay. - didn't work
ea - update_assignments.py
xmgr.py
   
ed current_directory_to_sparky.py - easier access to current directory:

'ed' makes a simbolic link to the current
working directory (you started sparky from)
in Save and Project directories called
a.Current_Direcory
   
ep  - print_selected_peak.py - assists work with the assignments: prints the position and the
assignment of the selected peak into the python shell: so
Evgenii can cut and paste into his assignment worksheets
eg -    grab_peaks.py - Grab peak ppm range
ey - select_peaks_group.py - Sparky extension to select a group peaks in the spectrum
using peak list in the text file. Format: first word in every string is considered to be
a peak name. 'Assignment' header line is ignored
   
cl - copylinewidth.py - my extension to help fitting of series of spectra: you fit first one adjusting everything
and then you copy linewidths and positions to other ones and fit only heights.

eR - Recalculate relaxation times by equation - 'recalculate_times.py'

ex - xeasy.py - updated extension for output of XEASY format compatible with GARANT and PINE assignment programs.

Conversion of Sparky-exported spectral regions into small PNG files for HTML notebooks

eX - export_peak_snapshot.py - Exports a rectangular region around a selected peak for record-keeping, which is convertable to PNG for HTML documents using dat2png Matlab program.

 


 

 

 

NOTES on new Sparky extension development

  1. Use 'new_extension.py' file for work, then you will be able to reload a code into Sparky without exiting it by er and run it by zz
  2. 'new_extension' module is called by run method so make sure you include the main funciton of the extension into this run() method
  3. If you need to look up any values in Sparky environment in the Python Shell (py) use
    1. import sparky
    2. sparky.session_list[0] - this is a Session instance that contains all the current data

Analysis of Sparky exported data 

add_peak_labels.py - add consequtively numbered labels to ?-? peaks from automatic peak picking


remove_residues.py - Takes an lt-list 'source_list' of residues from Sparky and removes from this list residues found in 'unwanted_list'. The result is recorded in 'cleaned_up_list' and is importable into Sparky with 'ey'.

chem_shift_change.py - calculation of chemical shift changes in spectra and generation of data arrays for titration curve analysis in MatLab

compare_protocols.py    -  compare differences between per-residue values
detect_overlap.py            - detect overlap analyzing Sparky lists
   
remove_assignment_suffix.py - removes 'C-H' or like suffix from residue names in all the file
                                                        of a directory given as a command line argument

Annn2nnn.py
- strips 1st letter from every line in the file so that the data
                            become importable to XMGR
   
LT_list_extract_column.py - extracts a column from Sparky LT list
   
es2xmgr.py - plotting of 'es' lists on XMGR as Height vs. Time
   
es2text.py - conversion of 'es' list into plain text 'x y dy' format with optional normalization
   
plot_data_Annn.py - plotting a number of ANNN residue lists on XMGR (-diff, -reldiff)
plot_data_NNN.py - same for just NNN format
plot_1vs2.py - plots one dataset versus the other with errors
   
   
   
etaxyfit_xmgr_read.py
EtaxyR2ratio2.py
EtaxyR2ratio3.py
duplicate_errors.py - Error analysis using duplicates in the time series
   
R1vsT_data_extract.py - in multiple datasets: extract data for the same residue to XMGR vs. parameter
sort_manual_list.py - Sorting of manually prepared list of residues
   
smart_average.py         - calculates AVERAGE and SIGMA of full and x% trimmed dataset
trim_mean_sigma.py      - (older version - works only with Annn residue format)
trim_by_errors.py           - trimming of data by size of error column values

difference.py     -     calculate difference between two datasets for residues found in both of them
dRex_calc.py     -     calculation of R2 difference between fast and slow pulsing datasets in rcCPMG
R1ro_R1_delta_calc.py     -     (ekn126) pairwise analysis of R1rho-R1 data (small and large offset)
R1ro_deltaRex_calc.py     -     (ekn128) Analysis of R1rho data combined with R1 data
   

 

rcCPMG R2 and simple Rex determination:
es2m.py
- extract peak heights saved as 'es' list into format compatible with R2fitting/ (Matlab) . The program works in conjunction with ek_rccpmg.py and ek_rccpmg15N pulse program to produce data matrices for fitting relaxation dispersions.

rcCPMG Rex fitting :
es2m_full.py - extract peak heights saved as 'es' list into format compatible with RexFitting/ (Matlab) .

concatenate_intensity_data.py - This program concatenates datasets obtained in different rcCPMG experiments but at the same temperature. It allows fitting together data obtained at with different CPMG frequency tables.

 

make_prot_seq_files.py - The program to produce PROT and SEQ files required for GARANT and PINE assignment programs


nmrPipe-directed data processing



see also 'Processing' directories !!!! (this may be not last versions)
==========================================================
adjust_phases_XYZ.py - incremental phasing of 3D-datasets
   
ProcessAllNoe.py - (old version)
ProcessNoeSeparately2.py - NOE processing with sophisticated error analysis
NOEratio.py - NOE relaxation data analysis
NOEratio_ini.py
NOEratio_lib.py
   
separate_fids2.py - separating interleaved fids
ProcessAll_interleaved.py - processing of interleaved data (with separate_fids2.py)
   
test_statistics.py - Error analysis: check statistics 'Palmer' statistics vs Gaussian simulation
xmgr2ps.py - the utility to convert all XMGR files in the directory into the PostScript format
   
pipe2text.py - conversion of 1D data to text format
   
add.py - adds two 3D experiments together: truncated dataset to full dataset
   
add_fids.py - adds multiple fids (repeats of the same experiment) together
    

ModelFree related 

extract_nonexch.py
trim_mean_sigma.py - trimming of the dataset (on particular column)
extract_common_residue_set.v2.py - finding intersection of supplied datasets
trim_by_errors.py - removal of data points with high errors from the dataset
   
   
R2R1_for_r2r1diffusion.py
R2R1ratio.py
   
mfout_diff.py
MFselect_checklist_before_run.p
MFselect_compare.py
MFselect_constants.py
MFselect_fixed_tensor.py
MFselect_flowcharts.py
MFselect_full_run.py
MFselect_lib_2.py
MFselect.py
mf_test.py
   
spectral_density.py - Reduced spectral density calculations
   
FMF_extract_model.v2.py - extract data from results of FastModelfree
FMF_extract_S2.v2.py - ""

MFselect_extract_model_S2.py - extract model number, S2 and dS2 data from pickled results of MFselect
    ( old version: MFselect_extract_S2.py )
MFselect_compare.py - compare pickled data from MFselect, plot temperature series for each residue
   
MFselect_extract_same_model_residue_set.py  - extracts data from multiple files so that only residues that have the same model in every one are retained
   

Calculations of chemical equilibria (for ITC) 

equilibria.py
equilibria2.py

Ha_calc.py - calculation of activation energies from rate constants at different temperatures and prediction of constants at other temperatures
   
   


Pymol modules

Pymol/ - collection of my old Pymol python programs

PymolLib.py - library of Pymol functions

pymol_start.pml - Run this in Pymol to set up paths for my PymolLib

Custom programs:

 


 

Relaxation data analysis

 

 

 


Protein sequence analysis