Duke-UNC Brain Imaging and Analysis Center
BIAC Forums | Profile | Register | Active Topics | Members | Search | FAQ
Save Password   Forgot your Password?
 All Forums
 Support Forums
 Cluster Support
 Submitting jobs to cluster
 New Topic  Reply to Topic
 Printer Friendly
Author  Topic Next Topic  

Junior Member

40 Posts

Posted - Aug 07 2009 :  1:32:28 PM  Show Profile  Visit vvs4's Homepage  Send vvs4 an AOL message  Reply with Quote
Hi, I have some pretty basic questions about submitting scripts to run on qsub. I keep getting directed to the wiki and haven't found the answers that I'm looking for, so I'm hoping for better luck here!

This is my first time attempting to write and run scripts, my first time with bash scripting period, so these should be pretty basic.

I tried running a simple script with qsub and got the following message:

bash: /opt/gridengine/bin/lx24-amd64/qsub: Permission denied

Question 1: Do I need to be granted access to qsub? It gives the same message even when I just give it a stripped down "qsub"

Question 2: I'm not sure if I am trying to run it from the right place, if that makes sense. The only way that I know how to access my data (which is on Goldman) in the SSH window is by logging into qinteract, so I am navigated to my Scripts folder in my Experimental directory (I assume I have to be here to run the scripts?

Question 3: Script preparation! What is the most popular text editor for writing scripts? Outside of qinteract I was using nedit, then once logged in with my data nedit didn't work so I found emacs, but then I wasn't sure if I needed to just save, or compile first and use that version. After saving, I ran chmod u+x filename on the script and it turned green... does that simply uncomment the first line #/bin/sh so it becomes runnable? Is it a .sh file now? Is typing that command the same thing as manually uncommenting it? I am basically wondering about the steps to get from text --> .sh

Question 4: Script Housing! I had assumed all scripts that are relevant to a particular experiment go in EXPERIMENT/Scripts. When I call qsub to submit the job does it know to look for my scripts in that location? Or do the scripts need to go somewhere in bin?

Question 5: Running on multiple subjects. In the example on the wiki at http://fourier.biac.duke.edu/wiki/doku.php/biac:cluster:submit?s=script

it looks like I can run many subjects with run_script_all, and then list the relevant names at the end. Where is the run_script_all functionality located? It told me that it couldn't be found, and then when I removed it it gave me the permission denied message.

Thanks for your help in advance. Once I figure out these basics I'll be able to be the go to person for my lab, which hopefully means fewer people poking around the forums!



Edited by - vvs4 on Aug 07 2009 1:33:51 PM

BIAC Staff

453 Posts

Posted - Aug 07 2009 :  2:21:57 PM  Show Profile  Reply with Quote
1) You have access to the cluster, which means you have access to qsub ... it sounds like you are trying to run scripts from the qinteract node. You can only submit to the cluster from the head node. Qinteract is just for testing and getting things set up.

2) see #1 and .. To submit a script you either need to have it in your home directory on the head node, or you can make a submission script that basically changes directory into your Scripts directory after its mounted on the processing node and executes from there. (ie: everything from the Template script, but with "cd $EXPERIMENT/Scripts" ... then your script name in "user script section")

3) nedit might not be installed on node4 .. but you can use kedit or kwrite (just the kde version instead of the gnu version nedit). chmod u+x changes the user permissions of the script to be executable, thats why you can run it ... type ls -l SCRIPTNAME to see the permissions of your script. That first line of your script (the shebang line) is what makes it a .sh script in your case, since thats what you've declared that it is.

4) see #2

5) in that example "run_script_all" is just the name of the user's job ... -N allows you to name your job. If you want to write something that you can use for multiple subjects, then set-up your script so that it takes input variables so that you can run it multiple times with just different arguments from the command line.

for instance:

if your script has SUBJ=$1 RUN=$2.
then you can submit a job: qsub -v EXPERIMENT=YOUREXP.01 scriptName.sh 12345 1 .. then qsub -v EXPERIMENT=YOUREXP.01 scriptName.sh 6789 1 and it will run the same script, just replacing anywhere you have $SUBJ or $RUN with what you entered on the command line.
Go to Top of Page

Junior Member

40 Posts

Posted - Aug 08 2009 :  1:21:11 PM  Show Profile  Visit vvs4's Homepage  Send vvs4 an AOL message  Reply with Quote
Cool... I figured out how to get the scripts running! I have been able to submit jobs, and now need some help with the script itself. I took the following steps in writing a script to convert dicom to bxh.

1) wrote the script with nedit at vvs4@head,
2) saved it as dicom2bxh.sh, also at the head
3) took the # away from the !/bin/sh on the first line
4) exited and ran chmod u+x dicom2bxh.sh.
5) submit it using qsub -v EXPERIMENT=FIGS.01 dicom2bxh.sh /Data/RawData/$SUBJECT/4/ BXHheader.bxh 040526121358
6) qstatall shows my job go from the waitlist, to running, then disappear.

After changing the script and the submit line many times, I'm frustrated enough to ask for help again, because they don't work! Here is my script (questions I have are indicated by directly below the line in question.

here I took out the "#" but do I need to keep it?

# -------- DICOM 2 BXH ---------

# This script is intended for converting raw dicom images into a .bxh header
# so that QA can be run to check for data quality!
# This script puts the BXH header in the same folder as the dicom

# -------- SUBMITTING JOB ON COMMAND LINE --------------

# > qsub -v EXPERIMENT=FIGS.01 dicom2bxh.sh /Data/RawData/$SUBJECT/4/ BXHheader.bxh Subject1 Subject2 Subject3
does it matter that my variable, BXHheader, starts with a capital letter? Is it unwise to specify the extension here?

# There are 2 USER sections
# 1. USER DIRECTIVE: If you want mail notifications when
# your job is completed or fails you need to set the
# correct email address.
# 2. USER SCRIPT: Add the user script in this section.
# Within this section you can access your experiment
# folder using $EXPERIMENT. All paths are relative to this variable
# By default all terminal output is routed to the " Analysis "
# folder under the Experiment directory i.e. $EXPERIMENT/Analysis
# To change this path, set the OUTDIR variable in this section
# to another location under your experiment folder
# eg: OUTDIR=$EXPERIMENT/Analysis/GridOut
# By default on successful completion the job will return 0
# If you need to set another return code, set the RETURNCODE
# variable in this section. To avoid conflict with system return
# codes, set a RETURNCODE higher than 100.
# eg: RETURNCODE=110
# Arguments to the USER SCRIPT are accessible in the usual fashion
# eg: $1 $2 $3
# The remaining sections are setup related and don't require
# modifications for most scripts. They are critical for access
# to your data

#$ -S /bin/sh
#$ -o $HOME/$JOB_NAME.$JOB_ID.out
#$ -e $HOME/$JOB_NAME.$JOB_ID.out
#$ -m ea
why are these lines commented out?

#Name of experiment whose data you want to access
EXPERIMENT=${EXPERIMENT:?"Experiment not provided"}

source /etc/biac_sge.sh

EXPERIMENT=${EXPERIMENT:?"Returned NULL Experiment"}

exit 32
echo "----JOB [$JOB_NAME.$JOB_ID] START [`date`] on HOST [$HOSTNAME]----"
# -- END PRE-USER --
# **********************************************************

# Send notifications to the following address
$ -M youremail@duke.edu
I didn't receive this email... another indication my job failed


# User script goes here

do I need to input my experiment here somehow?


# OUTDIR is where the output folder will go
# OUTPRE is the output prefix of the data


# ------- LONG SCRIPT ------------------

do I need to go to $EXPERIMENT/$OUTDIR or am I already in the experiment directory?

#we navigate to FIGS.01/Data/RawData/$SUBJECT/4/
#to run bxhabsorb so output is dropped here.

bxhabsorb --fromtype dicom *.dcm $OUTPRE


# **********************************************************
echo "----JOB [$JOB_NAME.$JOB_ID] STOP [`date`]----"

For the $SUBJECT variable I intend to take a lot of subject folder names from the command line, in a list. I was just testing it on one folder.

I originally had another variable, called $DICOMFILES that took a path ending in *.dcm from the command line, but decided to take it out because it seemed unnecessary given that we were already located in that folder.

Any help would be greatly appreciated! I really really want to get this working.

Thanks :)


Edited by - petty on Apr 22 2021 10:54:55 AM
Go to Top of Page

BIAC Staff

453 Posts

Posted - Aug 08 2009 :  4:05:07 PM  Show Profile  Reply with Quote

You definitely need to keep that '#' on the first line, thats part of the declaration. You also need to keep in in the email line (#$ -M), which would be why you didn't get an email.

I would just declare the OUTDIR in the script because calling $SUBJECT from the command line isn't going to work, since its not set yet ... doesn't get set until it reaches that line in the script. Just set your OUTDIR to $EXPERIMENT/Data/RawData/$SUBJECT/4/ inside the script (after your SUBJECT=$ line ) and leave it off the command line.

You need to keep the $EXPERIMENT in your cd command because once your script is passed through the grid engine, you are just logged into a node, into your home directory ... so if you want to change somewhere, you need that full path.
Go to Top of Page
   Topic Next Topic  
 New Topic  Reply to Topic
 Printer Friendly
Jump To:
BIAC Forums © 2000-2010 Brain Imaging and Analysis Center Go To Top Of Page
This page was generated in 0.14 seconds. Snitz Forums 2000