skip to primary navigationskip to content

Nietlispach Group

NMR spectroscopy of membrane proteins

Studying at Cambridge

 

Processing guidelines

Cambridge CS

Quick Start

Processing data

2D processing

Higher dimensions

Running the reconstruction

Using the GUI

Using the data viewer (Graph plot)

Interfacing with NMRPipe

 

Quick Start

Cambridge CS can be called with the following command:

cambridgecs [OPTION] [Script file]

The full options available can be seen in the help text:

cambridgecs -h

To launch the gui:

cambridgecs -g

To load an existing script file into the gui:

cambridgecs -g script.scr

To run an existing script in automatic mode (command  line only):

cambridgecs -a script.scr

 

Processing data

In non-uniformly sampled experiments, the first (direct/acquisition) dimension is always fully sampled. Consequently, this dimension should be processed using the Fourier Tranform, with traditional tools. Cambridge CS is able to process the direct dimension within the software package, but can also interface with NMRPipe (see below for details).

Subsequently the indirect dimensions are processed. In essence, the processing software replaces the missing points with zeros and then the CS reconstruction attempts to 'fill-in' these points with the correct values, thus generating the ideal spectrum. The result can then be used for subsequent processing in the Cambridge CS package (using Azara) or for export to NMRPipe.

In the following, full processing (i.e. direct and indirect dimensions with post-processing in Cambridge CS) is described.

Processing Overview

Cambridge CS allows a script-based or GUI-based processing approach. The GUI generates a processing script and starts the reconstruction, subsequently opening a data viewer to display the result. When using the GUI, at any time the 'Open Raw Script' button shows the current state of the script file enabling the command syntax to be viewed. Commands may also be written in the scripting window and the 'Sync text' button updates the GUI.

There are four sections to the GUI/data-processing scripts.

Data parameters - information about the data e.g. file size, location etc.
Pre-processing - direct dimension processing (optional)
Reconstruction - indirect dimension reconstruction (required)
Post-processing - post-processing commands available in the Azara package e.g. baseline correction (optional)

Test data is available in the test_data folder.

2D processing example

# General properties:
    directory                          ~/data/CS/CScode/test_data/2D
    outputFile                         102_cs_nus
    threads                            1
    rawDirectory                     102
    nusList                            vclist
                
  direct-dim:
      npts                            1024
  end_script
                 
  indirect-dims:
      npts                            80
      ndim                            1
  end_script
           
# Pre-processing parameters:
           
    pre_processing_script:
       complex
       brukerGroupPhase               71.625
       gaussian_sw                    20.0    1.0    10000.0
       zerofill                       1
       fft                            
        phase                          103.0    40.0    1.0
       reduce                          
       range                          220    481
   end_script
          
# Reconstruction parameters:
        
    reconstruction_script:
       reconstruction_flag            IHT
       stop_flag                      l2
        
       dim 2:
           zerofill                   1
           phase                      180.0    -360.0
           mask_ppmm                  
            sinebell                   90.0
       end_script
   end_script
            
# Azara post processing instructions:
           
    post_processing_script:
       script_com 2:
           base_poly                  8.0    0.0
       end_script
      
       script_com 1:
           base_poly2                 8.0    1.0    1.0    261.0
       end_script         
end_script

 

Important points:

The first section contains general data properties.

# General properties:
    directory                          ~/data/CS/CScode/test_data/2D
    outputFile                         102_cs_nus
    threads                            1
    rawDirectory                       102
    nusList                            vclist
                
  direct-dim:
      npts                            1024
  end_script
                 
  indirect-dims:
      npts                            80
      ndim                            1
  end_script

All data is treated as a pseudo-2D and the total number of real and imaginary points in the direct and all indirect dimensions is given. The number of indirect dimensions, ndim, must also be specified. Other important fields in this section are:

directory - directory containing the data; for Bruker data this is assumed to contain the data folder
rawDirectory - for Bruker data this is the folder containing the unprocessed data i.e. ser file
threads - the number of cpu threads to use. By default this is set to 1. '0' uses the maximum number of available threads.
outputFile - prefix for all output data
nusList - sampling list specifying the recorded points

 

The pre-processing script contains a series of commands to process the direct dimension.

# Pre-processing parameters: 
           
    pre_processing_script:
       complex
       brukerGroupPhase               71.625
       gaussian_sw                    20.0    1.0    10000.0
       zerofill                       1
       fft                            
        phase                          103.0    40.0    1.0
       reduce                          
       range                          220    481
   end_script

Commands are performed in a linear manner i.e. order matters. These commands follow the Azara processing style with the addition of the brukerGroupPhase command which takes the GRPDLY parameter found in Bruker acqus files. It is important to include the reduce command (after the Fourier transform) in order to discard the imaginary part which is not required for further processing.

The reconstruction script carries out reconstruction of the indirect dimensions.

# Reconstruction parameters: 
        
    reconstruction_script:
       reconstruction_flag            IHT
       stop_flag                      l2
        
       dim 2:
           zerofill                   1
           phase                      180.0    -360.0
           mask_ppmm                  
            sinebell                   90.0
       end_script
   end_script

Since the indirect dimensions are processed together, the order of commands is not important here. The type of reconstruction is specified first, either FT or various CS algorithms available with the software (currently IHT, IST, IRL1). In the example given here, iterative hard thresholding (IHT) is used. A choice of stopping criteria (stop_flag) is available. The default, l2, works well. Commands for each of the indirect dimensions again follow the Azara style. Various window functions are available, zerofilling and data manipulation options (mask_ppmm, conjugate etc.) to handle different forms of data recording. It is important to set the correct phasing for each of the indirect dimensions. Additionally if (0,0) phase is used, scaleT0 should be set to 0.5 to correct the first point scaling.

The best way to determine the correct indirect dimension parameters is to process with the FT first. This is set by changing Reconstruction type (reconstruction_flag) to FT. The data can then be processed quickly and the phasing etc. checked before subsequently reconstructing the data using CS.

Finally the post-processing section specifies any data processing required after reconstruction.

# Azara post processing instructions: 
           
    post_processing_script:
       script_com 2:
           base_poly                  8.0    0.0
       end_script
      
       script_com 1:
           base_poly2                 8.0    1.0    1.0    261.0
       end_script         
    end_script

In the example given above, the output from the CS reconstruction is the frequency domain data and only baseline correction is applied, again in the Azara syntax. It is also possible to use the time-domain data (useTimeData); since the missing points have been reconstructed, full FT processing (using Azara commands) can be applied at this stage, in a similar manner to the direct dimension processing above. Available options can be seen in the GUI and further details in the manual or on the Azara website.

Higher dimensions

To extend to more indirect dimensions, the ndim option is changed in the first section (number of indirect dimensions) and additional processing parameters need to be specified in the reconstruction script section for each additional dimension. If setting up the script using the GUI, increasing ndim will automatically increase the number of dimensions available in the reconstruction script.

An example of a 3D processing script is shown below:

# General properties:
    directory                           ~/data/CS/CScode/test_code/3D/HNCO
    outputFile                          550_cs
    threads                             1
    rawDirectory                        550
    nusList                             vclist

    direct-dim:
        npts                            1024
    end_script

    indirect-dims:
        npts                            1400
        ndim                            2
        interlace                       2
    end_script

# Pre-processing parameters:

    pre_processing_script:
        complex                        
        brukerGroupPhase                71.625
        sinebell                        60.0
        zerofill                        3
        fft                            
        phase                           103.0    32.0    0.0
        reduce                         
        range                           1050    1661
    end_script

# Reconstruction parameters:

    reconstruction_script:
        reconstruction_flag             IHT
        stop_flag                       l2

        dim 2:
            zerofill                    2
            phase                       90.0    0
            sinebell                    60.0
        end_script

        dim 3:
            zerofill                    1
            phase                       0    0
            sinebell                    60.0
        end_script
    end_script

# Azara post processing instructions:

    post_processing_script:
        script_com 2:
            reverse                    
            base_poly                   6.0    0.0
        end_script

        script_com 3:
            reverse                    
            base_poly                   6.0    0.0
        end_script

        script_com 1:
            base_poly2                  8.0    1.0    1.0    612.0
        end_script
    end_script

 

Running the reconstruction

Once the script has been set up it can be run by pressing the button 'Process current script' in the GUI. This will automatically save the parameters in a .scr file with the prefix set by the outputFile option. Alternatively, the script can be saved manually using the 'Save file' button and then code run from the command line using:

cambridgecs -a script_name.scr

This is useful for longer reconstructions.

If processing via the GUI, press 'Process current script' to start the reconstruction and the GUI will automatically open a window showing the reconstruction progress. Once the reconstruction is completed, the standalone 'Graph plot' GUI is opened to display the reconstructed data.

 

Using the GUI

Calling the GUI with

cambridgecs -g

results in a blank GUI window shown below:

cambridgecs_gui

The four sections of the script, 'Data parameters', 'Pre-processing script', 'Reconstruction script' and 'Post-processing script', make up the four quadrants of the GUI.

The various elements of the script can be built up by filling in the text boxes, adding parameters from the 'Extra Params' dropdown menus or, in the Pre- and Post-processing sections, by clicking the green cross to reveal the list of available commands. Options from the 'Extra Params' sections can be removed at any point by unticking the option in the dropdown menu, while options added in the Pre- and Post-processing sections will have a red cross next to them which allows them to be removed.

At any time the script can be viewed by clicking 'Open script editor'. Options can be edited in the script editor and clicking the 'Sync text' button at the bottom of the script editor will sync any changes with the GUI.

A completed script in the GUI will look as follows (shown for the 2D example given above):

2d_cambridgecs_gui

Once the script is completed press the 'Process current script' button to start the reconstruction. Starting the reconstruction automatically saves the script with the prefix given by the 'Output filename' parameter followed by '.scr'. The script can also be saved to a different filename using the 'Save file' button.

A saved script can be loaded directly into the GUI using the 'Load file' button, or by calling the programme with the script as an argument:

cambridgecs -g script.scr

 

Using the Data Viewer (Graph plot)

The standalone data viewer, 'Graph plot' enables the user to quickly view the output from Cambridge CS reconstructions. The Graph plot window looks as follows:

graphPlot

 

The viewer opens automatically if processing in the Cambridge CS GUI or can be launch from the command line as follows:

graphPlot spectrum.spc.par

where spectrum.spc.par is an Azara style parameter file (which is used for the output format of Cambridge CS).

The viewer enables a number of simple spectrum manipulations:

Interfacing with NMRPipe

CambridgeCS can be used in conjunction with NMRPipe to reconstruct undersampled data. Use the script nusExpand.tcl from NMRPipe to replace the missing points with zeros (more details on NMRPipe NUS processing can be found here). In this example, ser is the undersampled, raw Bruker data, vclist is the list of sampled points, '-sampleCount' is the number of sampled points and ser_full is the zero-padded data.

nusExpand.tcl -mode bruker -sampleCount 350 -off 1 \
 -in ../550/ser -out ../550/ser_full -sample vclist

The data can then be converted using a standard bruk2pipe script. For example:

bruk2pipe -in ../550/ser_full \
  -bad 0.0 -ext -noaswap -AMX -decim 16 -dspfvs 12 -grpdly -1  \
  -xN              1024  -yN                64  -zN                96  \
  -xT               512  -yT                32  -zT                48  \
  -xMODE            DQD  -yMODE  Echo-AntiEcho  -zMODE        Complex  \
  -xSW        10000.000  -ySW          2530.36  -zSW          1666.667 \
  -xOBS         500.132  -yOBS          50.684  -zOBS         125.780  \
  -xCAR           4.773  -yCAR         120.376  -zCAR         175.972  \
  -xLAB              HN  -yLAB             15N  -zLAB             13C  \
  -ndim               3  -aq2D         Complex                         \
  -out ./fid/test%03d.fid -verb -ov

Subsequently, the data can be processed in the direct dimension. The output should be a single file. Note that CambridgeCS processes the data 'in place' so there is no need to transpose the data after the direct dimension FFT, i.e. the direct dimension remains the first dimension.

xyz2pipe -in fid/test%03d.fid -x  \
| nmrPipe  -fn SOL                                  \
| nmrPipe  -fn SP -off 0.5 -end 0.98 -pow 2 -c 0.5  \
| nmrPipe  -fn ZF -zf 3                             \
| nmrPipe  -fn FT                                   \
| nmrPipe  -fn PS -p0 151  -p1 0.0 -di               \
| nmrPipe  -fn EXT -x1 1050Pts -xn 1660Pts -sw                        \
 -ov -out test.nus

At this point CambridgeCS can be used to reconstruct the data in the indirect dimensions. A script file is generated, this time omitting the 'Pre-processing parameters', since the direct dimension is already processed. The 'nmrPipe' flag and 'head' size (512) are included, as well as the 'max_points' in dim2 and dim3. The script can be run via the GUI, or from the command line.

# General properties:
    directory                           ~/data/CS/CScode/test_code/3D/HNCO/NMRPipe
    outputFile                          550_pipe_cs
    threads                             1
    dataInput                           ~/data/CS/CScode/test_code/3D/HNCO/NMRPipe/test.nus
    rawDirectory                        ../550
    nusList                             ../NMRPipe/vclist_coordinate
    nmrPipe
    head                                512

    direct-dim:
         npts                            611
    end_script

    indirect-dims:
         npts                            6144
         ndim                            2
   end_script


# Reconstruction parameters:
     reconstruction_script:
       reconstruction_flag             IHT
stop_flag l2

       dim 2:
            zerofill                    2
            phase                       0    0
            scaleT0                     0.5
            max_points                  64
            sinebell                    60.0
       end_script

       dim 3:
           zerofill                    1
           phase                       0    0
           scaleT0                     0.5
           max_points                  96
           sinebell                    60.0
       end_script

    end_script

 

The script generates an output file *_recon.spc.par which can be viewed with Graph Plot. In addition a file *.ft1 is created, which contains reconstructed time-domain data in the indirect dimension(s). This can be further processed with nmrPipe. For example:

nmrPipe -in 550_pipe_cs.ft1 -verb \
| nmrPipe -fn TP \
| nmrPipe -fn SP -off 0.5 -end 0.98 -pow 2 -c 0.5 \
| nmrPipe -fn ZF -zf 2 -auto                      \
| nmrPipe -fn FT                                  \
| nmrPipe -fn PS -p0 0 -p1 0 -di               \
| nmrPipe -fn TP \
| pipe2xyz -out ft2/test%05d.ft2 -x -ov

xyz2pipe -in ft2/test%05d.ft2 -z -verb \
| nmrPipe -fn SP -off 0.5 -end 0.98 -pow 1 -c 0.5 \
| nmrPipe -fn ZF -zf 1 -auto                      \
| nmrPipe -fn FT -neg                        \
| nmrPipe -fn PS -p0 0 -p1 0 -di               \
| pipe2xyz -out recon/test%03d.ft3 -z -ov \

#Output as a single file for viewing in CCPN Analysis
xyz2pipe -in recon/test%03d.ft3 -x -verb \
| nmrPipe -ov -out 550_pipe_cs.ft3

/bin/rm -rf *.dat

#Create 2D projections of the data.
proj3D.tcl -in recon/test%03d.ft3

/bin/rm -rf ft2

The scripts can easily be combined into a single processing script, with the command 

set dSize = `getParm -in test.nus -parm NDSIZE -dim CUR_XDIM -fmt %.0f`

used to find the number of points in the direct dimension of the experiment after direct dimension processing. This can be passed to the reconstruction with the flag --npts0:

cambridgecs -a ./550_pipe_cs.scr --npts0 $dSize

The full script is shown below:

#!/bin/csh

xyz2pipe -in fid/test%03d.fid -x  \
| nmrPipe  -fn SOL                                  \
| nmrPipe  -fn SP -off 0.5 -end 0.98 -pow 2 -c 0.5  \
| nmrPipe  -fn ZF -zf 3                             \
| nmrPipe  -fn FT                                   \
| nmrPipe  -fn PS -p0 151  -p1 0.0 -di               \
| nmrPipe  -fn EXT -x1 1050Pts -xn 1660Pts -sw                        \
-ov -out test.nus

set dSize = `getParm -in test.nus -parm NDSIZE -dim CUR_XDIM -fmt %.0f`
echo "Direct dimension has $dSize points"

cambridgecs -a ./550_pipe_cs.scr --npts0 $dSize
nmrPipe -in 550_pipe_cs.ft1 -verb \
| nmrPipe -fn TP \
| nmrPipe -fn SP -off 0.5 -end 0.98 -pow 2 -c 0.5 \
| nmrPipe -fn ZF -zf 2 -auto                      \
| nmrPipe -fn FT                                  \
| nmrPipe -fn PS -p0 0 -p1 0 -di               \
| nmrPipe -fn TP \
| pipe2xyz -out ft2/test%05d.ft2 -x -ov

xyz2pipe -in ft2/test%05d.ft2 -z -verb \
| nmrPipe -fn SP -off 0.5 -end 0.98 -pow 1 -c 0.5 \
| nmrPipe -fn ZF -zf 1 -auto                      \
| nmrPipe -fn FT -neg                        \
| nmrPipe -fn PS -p0 0 -p1 0 -di               \
| pipe2xyz -out recon/test%03d.ft3 -z -ov \

xyz2pipe -in recon/test%03d.ft3 -x -verb \
| nmrPipe -ov -out 550_pipe_cs.ft3

/bin/rm -rf *.dat

proj3D.tcl -in recon/test%03d.ft3

/bin/rm -rf ft2