SURA has defined a common set of software
that should be available on all SURA server systems. This software includes
both services and clients for those services and we provide a convenient way to
install this software. The SURAgrid software stack leverages the Virtual Data Toolkit (VDT) efforts, so you
may see references to the VDT in this document and during the installation
process. We are collaborating in the use and development of this stack with the
TIGRE project.
To access the formal SURAgrid installation documentation click here: Adding Resources to
SURAgrid.
These instructions assume that you will be performing the installation as a
root user.
The SURAgrid software stack consists of the following components:
VDT supports a variety of operating system and OS versions. Please make
sure your platform is one of the supported
operating systems. The SURAgrid software stacks require the following
underlying software to be available:
The disk space requirements vary per platform but you should have no
problems if you have 1-2 GB of free disk space.
1) Log on as root, source the existing setup.sh for the old installation.
2) Shut down existing
services using:
> vdt-control --offMake sure everything
is really shut down; do a:
> ps -efwww to check for stray processes if needed and
kill anything that references the old software
installation manually if needed.
3) Log out and back in as root. Make the new
blank installation area:
> mkdir /usr/local/suragrid_upgrade_number (ex: suragrid_1_8_1)4)
Set the environmental variable OLD_VDT_LOCATION to point to the old area.
5) Proceed with a new installation in the
new area as per the normal installation instructions,
including those for the batch adapter for the local
scheduler.
One should review the post-install/README
and do various other sanity checks for any local
configuration information that may need to be tuned to
the local conditions and setup
Before you
begin the following instructions, please create the globus user and group
(consult your operating system documentation for user creation instructions)
and set the default umask to 022:
> umask 022The SURAgrid
software stack is installed and managed with Pacman. For this release, we
require version 3.19 or greater of Pacman. You can download the latest version
of Pacman from the Pacman
web site. Go to the directory where you want the Pacman software to be
installed and unpack the tarball. In this example, we choose to unpack the
Pacman software into /usr/local. Also,
you must make sure previously downloaded versions of Pacman are deleted before
beginning this process.
> cd /usr/local > tar -zxvf pacman-latest.tar.gzThen set up
your environment to use pacman (replace * with the version you just downloaded
determined by seeing where the tar file unpacked itself):
> cd /usr/local/pacman-3.* > source setup.shThis example
assumes you're using the Bourne shell or Bash. If you're using the C shell you
should source setup.csh instead. Pacman requires at least version 2.2 of
Python. If this requirement is not satisfied, then Pacman will attempt to
download and install Python 2.2 in its own directory.
If you are installing the VDT server and
you have an external Condor installation that you would like to use in place of
the Condor that is installed with VDT you can specify it by setting two
environment variables described below before installing the VDT.
Doing so will produce the following
effects:
$VDT_LOCATION/setup.[c]sh will set PATH, MANPATH, CONDOR_LOCATION, CONDOR_CONFIG
to point to the necessary locations in your external Condor installation /etc/xinetd.d/globus-gatekeeper)
and Globus condor jobmanager ($VDT_LOCATION/globus/lib/perl/Globus/GRAM/JobManager/condor.pm)
will be set up to submit jobs to your external Condor $VDT_LOCATION/globus/etc/grid-info-resource-ldif.conf)
will be setup to report information about the external Condor. To specify an external Condor
installation you need to set the following two environment variables before
installing the VDT:
VDTSETUP_CONDOR_LOCATION: the location of your Condor installation (e.g. /opt/condor). The Condor bin/, sbin/, etc/, lib/... directories
should be directly under this location. VDTSETUP_CONDOR_CONFIG (optional): The location of your Condor configuration file (if
non-standard). Default is $VDTSETUP_CONDOR_LOCATION/etc/condor_config.
For example, if Condor is installed in /usr/local/condor
and the condor_config file is in /usr/local/etc you
could do:
For BASHexport VDTSETUP_CONDOR_LOCATION /usr/local/condor
export VDTSETUP_CONDOR_CONFIG /usr/local/etc/condor_configpacman -get VDT
For C-shell:setenv VDTSETUP_CONDOR_LOCATION /usr/local/condor
setenv VDTSETUP_CONDOR_CONFIG /usr/local/etc/condor_configpacman -get VDTMake and go
to the root directory where you want to install the SURAgrid server software
stack. For the remaining instructions, we will use /usr/local/suragrid as the
root directory. Then use pacman to begin installing the server software stack:
> mkdir /usr/local/suragrid > cd /usr/local/suragrid > pacman –get http://omnius.hpcc.ttu.edu/suragrid:SURAgridIf you get a
Pacman error about an unsupported platform and your platform is listed as
supported on the VDT website, try a "pacman -pretend-platform" option
choosing one of the supported platforms that is close to your operating system
environment. If you encounter problems or have other questions, please send an
e-mail message to the SURAgrid Support
e-mail list.
The
installation should begin with Pacman asking you questions and downloading a
relatively large number of packages. The time to complete the installation will
vary depending on your network connection and the available bandwidth at the
VDT server. There will be prompts for the you to answer some questions:
Do you want to add http://omnius.ttu.edu/suragrid/SURAgrid.pacman to [trusted.caches]?Do you want to add http://vdt.cs.wisc.edu/vdt_181_cache to [trusted.caches]?Do you agree to the licenses?Would you like to enable the Condor batch system to run automatically?Would you like to enable Globus Gatekeeper daemon to run automatically?Would you like to setup daily rotation of VDT log files?Would you like to enable Globus GRIS daemon to run automatically?Would you like to enable Globus GridFTP daemon to run automatically?Would you like to enable Globus web-services container to run automatically? > source /usr/local/suragrid/setup.shAfter the
installation is complete, there are a number of post-install configure steps
that you need to perform before the SURAgrid server software is fully
functional.
If you haven't
already done so, you need to obtain credentials for your host. The SURAgrid
authentication and authorization infrastructure is based on a two-tier PKI
approach coupled with an optional LDAP-based PAM callout. See the SURAgrid PKI Bridge
Certification Authority and User Management System pages for more
information. You should also obtain user grid credentials for yourself so that
you can test your installation.
The 2 files you
need to install on the server are hostcert.pem and hostkey.pem. Copy these
files to /etc/grid-security and make sure they are owned by the globus user.
These files will be used by GSIOpenSSH and GridFTP. In /etc/grid-security, you
also need to copy hostcert.pem to containercert.pem and hostkey.pem to
containerkey.pem if they were not already in place at teh time of your SURAgrid
server stack installation done above. The container*.pem files are used by the
Globus web services container. The container*.pem files must be owned by the
globus user because the Globus web services container will run as the globus
user.
As mentioned
above, you have the option of either using the SURAgrid LDAP callout to control
local account mapping and authorization or simply setting up a grid-mapfile to
do so. Such mapping is required to associate the subject distinguished name for
a particular user in their X.509 certificate to a local Unix account.
To get
started, you can simply put an entry for your own personal cedrtificate into a
local grid-mapfile. This file is located at /etc/grid-security/grid-mapfile. So
that you can perform testing, create the /etc/grid-security/grid-mapfile file.
In this file, create a line containing your certificate subject DN (run
grid-cert-info -subject) in quotes and your username. For example:
"/C=US/O=UTAustin/OU=TACC/CN=John Doe/UID=jdoe" jdoeThis will
map the indicated user identified by the name /C=US/O=UTAustin/OU=TACC/CN=John
Doe/UID=jdoe to the local Unix user jdoe.
For further
information, see the grid-mapfile
section of the SURAgrid PKI Bridge Certification Authority and User
Management System pages. Note that this topic is undergoing some evolution
within SURagrid, and we hope in particular to provide a mechanism for a
fully-accredited approach in cooperation with the International
Grid Trust Federation soon.
The GSI version of SSH allows users to
ssh in to your system using their grid credentials. They do not have to provide
a password and are automatically mapped to their local assigned grid userid
upon gsissh login. To set this up, you must enable your
gsissh server. The easiest way to do these steps after installing the
SURAgrid package is to edit the $VDT_LOCATION/post-install/sshd file to add any
options you may need (for example, specifying the port number using the
"-p nnnnn" option), then (once only) issue the command:
vdt-register-service --name gsisshd --type init --enable --protocol tcp --init-script $VDT_LOCATION/post-install/sshd
After doing
this step, the gsisshd service can be started or stopped using the commands
vdt-control --on gsisshdorvdt-control --off gsisshdYou should
then test your
gsissh server or proceed directly to using it by simply changing to your
user account, setting up either the SURAgrid client or server package, getting
a grid proxy and then connecting to your server resource using the command
"gsissh (your server) -p nnnnn" where nnnnn is the port you selected
during the configuration step above.
The Globus GridFTP
provides high-performance file movement between SURAgrid systems. The GridFTP
server runs out of xinetd and no configuration is required after the above
installation. You can test your GridFTP installation using globus-url-copy
or uberftp.
We recommend
running the Globus web services container as an ordinary user for increased
security. The typical userid used is 'globus' and you should create this
account on your system if it does not already exist.
The Web
Services GRAM (WS-GRAM) component allows remote users to execute applications
on your system.
In order to
configure WS-GRAM, sudo must be available. The Globus web services container
executes as the globus user for improved security. But, when a user submits an
application to execute, the WS-GRAM service needs to become that user to submit
and manage the application. This is accomplished using sudo.
An easy way
to configure sudo is to give the globus user permission to run the commands it
needs to as any other user than root. This is accomplished by adding the
following to your /etc/sudoers file (replace the with your installation
directory). Note that you must give the EXACT path to the respective components
in the additions below; soft-links will not work, so give the exact path. Use
visudo to make these changes:
Runas_Alias GLOBUSUSERS = ALL, !rootglobus ALL=(GLOBUSUSERS) \ NOPASSWD: /usr/local/suragrid/globus/libexec/globus-gridmap-and-execute \ -g /etc/grid-security/grid-mapfile \ /usr/local/suragrid/globus/libexec/globus-job-manager-script.pl *globus ALL=(GLOBUSUSERS) \ NOPASSWD: /usr/local/suragrid/globus/libexec/globus-gridmap-and-execute \ -g /etc/grid-security/grid-mapfile \ /usr/local/suragrid/globus/libexec/globus-gram-local-proxy-tool *By default,
the WS-GRAM can start applications only on the host it is running on. However,
when WS-GRAM is running on a cluster, it needs to submit the application to the
batch scheduler. This support can currently be provided for the Condor, LSF,
and PBS scheduling systems. Please contact the SURAgrid Support
e-mail list if you require support for a different scheduler. To set up
WS-GRAM to interact with your scheduler, execute one of the following commands.
(NOTE: Double-check that the cache is the same one your installation used in
the above step, and adjust the appropriate command below before issuing it if
not!):
> pacman -get http://vdt.cs.wisc.edu/vdt_181_cache:Globus-WS-Condor-Setup or> pacman -get http://vdt.cs.wisc.edu/vdt_181_cache:Globus-WS-LSF-Setup or> pacman -get http://vdt.cs.wisc.edu/vdt_181_cache:Globus-WS-PBS-SetupTo set the
globus tcp port range to match your local preferred values, or to set any other
special local conditions for your installation, you can put commands into the
files $VDT_LOCATION/vdt/etc/vdt-local-setup.sh AND .csh (formatted
appropriately for each shell), as follows:
In $VDT_LOCATION/vdt/etc/vdt-local-setup.sh export GLOBUS_TCP_PORT_RANGE="40000,50000"export GLOBUS_TCP_SOURCE_RANGE="40000,50000" AND in $VDT_LOCATION/cat vdt/etc/vdt-local-setup.cshsetenv GLOBUS_TCP_PORT_RANGE "40000,50000"setenv GLOBUS_TCP_SOURCE_RANGE "40000,50000"Add any
other commands as needed to handle variations that may apply to your local
cluser environment. Once such local conditions have been set, a
"vdt-control --off" followed by "vdt-control --on" should
start your SURAgrid services with the local settings applied. Use
"vdt-control --list" to see what you have installed and whether it is
enabled to respond to vdt-control --on and --off control commands.
You can now
start your Globus services, including the container that contains the WS-GRAM
service, by executing
> vdt-control --onNoe: If you
had pre-existing services of the same name left over from previous installations,
you can add the parameter "--force" to the above vdt-control command
the first time you issue it. The previous services will be renamed and retained
but not started by the VDT once this is done. You may want to go through one
cycle of "vdt-control --off --force" followed by "vdt-control
--on" to get to a known state your first cycle through. After this,
"vdt-control --off" or "--on" should be enough to turn off
or on your grid services.
You can test
your WS-GRAM installation using the globusrun-ws
program.
> vdt-control --on> vdt-control --off> cd /usr/local/suragrid_1_8_1/post-install/certificates>./install_ca_certificates --root or>./install_ca_certificates --local
>/usr/local/suragrid_1_8_1/globus/bin/grid-cert-request>/usr/local/suragrid_1_8_1/vdt/setup/setup-cert-request>vi /usr/local/suragrid_1_8_1/condor/local.erasmus/condor_config.local
>/usr/local/suragrid_1_8_1/vdt/setup/configure_mds>/usr/local/suragrid_1_8_1/globus/setup/globus/setup-simple-ca>/usr/local/suragrid_1_8_1/globus/setup/globus_simple_ca_HASH_setup/setup-gsi -default -nonroot>cd /etc/grid-security>cp hostkey.pem containerkey.pem>cp hostcert.pem containercert.pem>chown globus: containerkey.pem containercert.pemRunas_Alias GLOBUSUSERS = user1, user2globus ALL=(GLOBUSUSERS) \ NOPASSWD: /usr/local/suragrid_1_8_1/globus/libexec/globus-gridmap-and-execute \-g /etc/grid-security/grid-mapfile \/usr/local/suragrid_1_8_1/globus/libexec/globus-job-manager-script.pl *globus ALL=(GLOBUSUSERS) \NOPASSWD: /usr/local/suragrid_1_8_1/globus/libexec/globus-gridmap-and-execute \-g /etc/grid-security/grid-mapfile \/usr/local/suragrid_1_8_1/globus/libexec/globus-gram-local-proxy-tool *>Runas_Alias GLOBUSUSERS = ALL, !rootIf you have
any problems installing the SURAgrid software, please contact us using the SURAgrid Support
e-mail list.
There is a
VDT Discuss and Announce list which might be helpful if you have specific
advanced usage scenarios. Please see the VDT Support page for a addtional
information.