BeSTGrid Notes

Remove service in Debian

2011-06-17T16:04:00.001+12:00

Removing Unwanted Startup Files or Services in Debian

Under Debian Linux startup files are stored in /etc/init.d/ directory and symbolic linked between /etc/rcX.d/ directory exists. Debian Linux uses System V initialization scripts to start services at boot time from /etc/rcX.d/ directory. Debian Linux comes with different utilities to remove unwanted startup file:

(A) rcconf

This tool configures system services in connection with system runlevels. It turns on/off services using the scripts in /etc/init.d/. Rcconf works with System-V style runlevel configuration. It is a TUI(Text User Interface) frontend to the update-rc.d command.

Install rcconf in Debian

#apt-get install rcconf

To start rconf, login as root user and type rcconf

# rcconf

Select the service you would like to enable or disable.

(B) sysv-rc-conf

sysv-rc-conf provides a terminal GUI for managing "/etc/rc{runlevel}.d/" symlinks. The interface comes in two different flavors, one that simply allows turning services on or off and another that allows for more fine tuned management of the symlinks. Unlike most runlevel config programs, you can edit startup scripts for any runlevel, not just your current one.

Install sysv-rc-conf in debian

#apt-get install sysv-rc-conf

This will install sysv-rc-conf.Now you need to run the following command

# sysv-rc-conf

Select the service you would like to enable or disable.

Both sysv-rc-conf and rcconf are best tools to use on Remote Debian Linux or when GUI is not available

You can also use update-rc.d script as follows (update-rc.d removes any links in the /etc/rcX.d directories to the script /etc/init.d/service):

# update-rc.d -f {SERVICE-NAME} remove

For example to stop xinetd service you can type command as follows:

# update-rc.d -f xinetd remove

Manage quotas on AIX

2011-05-20T16:11:00.004+12:00

Check quotas
quotacheck -vug /data/ecology

To edit User Limit Classes
j2edlimit -u /data/ecology

To assign class three to user 15003 for filesystem Ecology
j2edlimit -a 3 -u 15003 /data/ecology

Check quotas
quotacheck -vug /data/ecology

List quota for user 15003
quota -u 15003

Fix Kerberos Authentication

2010-09-23T17:26:00.001+12:00

authconfig --useshadow --enablemd5 --enablekrb5 --krb5realm=EC.AUCKLAND.AC.NZ --krb5kdc=kerberos.ec.auckland.ac.nz --krb5adminserver=kerberos.ec.auckland.ac.nz --updateall

List all files in DataStorage with their sizes

2010-03-31T17:08:00.002+13:00

find /data -fstype jfs2 2>/dev/null -type f -exec ls -lau {} \;|awk '{user=$3; group=$4; size=$5; date=$6; month=$7; time=$8; $1=""; $2=""; $3=""; $4="";$5="";$6="";$7="";$8="";print user,group,size,date,month,time,$0}' > file-size-access-2010-03-25

Output is like this:

cogneuro bestgrid 1275692 11 Mar 20:06 /data/cogneuro/Permanent Folder from M Drive/Software/Win_View/Coherence.pdf

To convert to comma delimiter file use in CentOS:

sed -r 's/ /,/;s/ /,/;s/ /,/;s/ /,/;s/ /,/;s/\s+/,/' file-size-access-2010-03-25 > file-size-access-2010-03-25.csv

NFS connection to VMWare

2010-01-28T11:49:00.002+13:00

To connect NFS share to VMWare as a DataStorage
VMkernel Port with external IP should be created and connected to any virtual switch with external NIC connected.

Access Control Lists (ACLs) in AIX

2010-01-28T11:33:00.003+13:00

Brian Browning requested to make his share accessible by him and Sharon with new UIDs and from old server with old UID. Old UID was 1016 and group 1000.
New UIDs (341808 and 24713) should have full access to the share.

Commands:

- to see ACL for object:
aclget /data/stat/browning/
output
*
* ACL_type AIXC
*
attributes:
base permissions
owner(1016): rwx
group(1000): ---
others: ---
extended permissions
enabled
permit rwx u:341808
permit rwx u:24713

- to edit ACL
set EDITOE variable:
export EDITOR=/usr/bin/vi
edit ACL:
acledit -v /data/stat/browning/

- to propogate new rights to other objects:
aclget /data/stat/browning/ | aclput -v -R /data/stat/browning/

Sakai Stats

2009-11-02T10:40:00.007+13:00

[root@sakai backup]$ cat sakaistat.sh
mysql -pSakaiMySQL sakai -t < /var/backup/sakaistat.sql > /var/backup/sakaistat.txt
mail -s "Sakai usage stats" a.kharuk@auckland.ac.nz < /var/backup/sakaistat.txt

[root@sakai backup]$ cat sakaistat.sql
select "Total Users" as Property, count(user_id) as Amount from SAKAI_USER
union
select "System Users" as Property, count(user_id) as Amount from SAKAI_USER where type='system'
union
select "Registered users with e-mails" as Property, count(user_id) as Amount from SAKAI_USER where email is not null and type='registered'
union
select "Registered users without e-mails" as Property, count(user_id) as Amount from SAKAI_USER where email is null and type='registered'
union
select "Guests with e-mails" as Property, count(user_id) as Amount from SAKAI_USER where email is not null and type='guest'
union
select "Guests without e-mails" as Property, count(user_id) as Amount from SAKAI_USER where email is null and type='guest'
union
select "--------------------------------" as Property, 0 as Amount
union
select "Total sites" as Property, count(site_id) as Amount from SAKAI_SITE
union
select "Total Projects" as Property, count(site_id) as Amount from SAKAI_SITE where type="project"
union
select "Total My Workspaces" as Property, count(site_id) as Amount from SAKAI_SITE where title="My Workspace";

select a.month, ifnull(a.registered_with," ") as reg_email, ifnull(b.registered_without," ") as reg_noemail, ifnull(c.guests," ") as guests, ifnull(d.projects," ") as projects, ifnull(e.workspaces," ") as workspaces from
(select concat(year(createdon), "-", lpad(month(createdon),2,"0")) as month, count(user_id) as registered_with
from SAKAI_USER where type="registered" and email is not null group by year(createdon),month(createdon)) as a
left join
(select concat(year(createdon), "-", lpad(month(createdon),2,"0")) as month, count(user_id) as registered_without
from SAKAI_USER where type="registered" and email is null group by year(createdon),month(createdon)) as b
on a.month=b.month left join
(select concat(year(createdon), "-", lpad(month(createdon),2,"0")) as month, count(user_id) as guests
from SAKAI_USER where type="guest" group by year(createdon),month(createdon)) as c
on a.month=c.month left join
(select concat(year(createdon), "-", lpad(month(createdon),2,"0")) as month, count(site_id) as projects
from SAKAI_SITE where type="project" group by year(createdon),month(createdon)) as d
on a.month=d.month left join
(select concat(year(createdon), "-", lpad(month(createdon),2,"0")) as month, count(site_id) as workspaces
from SAKAI_SITE where title="My workspace" group by year(createdon),month(createdon)) e
on a.month=e.month;

Script file is sakai.bestgrid.org:/var/backup/sakaistat.sql

Sakai 2.6 installation

2009-10-19T15:34:00.001+13:00

- create sakai db:
-- CREATE DATABASE sakai CHARACTER SET utf8 COLLATE utf8_general_ci;
-- GRANT ALL ON sakai.* TO 'Sakai'@'localhost' IDENTIFIED BY 'Sakai#1' ;
- config sakai.properties

TSM backups config

2009-08-17T11:26:00.001+12:00

From: Roderic Campbell
Sent: Monday, June 08, 2009 3:48 PM

Andrey,

Due to ongoing problems with this backup, the best solution is to completely de-install
and then re-install the TSM client software on your client node.

Info you’ll need:

Your TSM-registered Node Name: DATA.BESTGRID.ORG

TSM server; D1PTSHSM1-BR

TCPadminport 1505
TCPPort 1500

TCP/IP Name: data.bestgrid.org
TCP/IP Address: 10.19.96.41

If you have lost your pwd, please let me know and I’ll reset it.

rgds,

Rod Campbell
TSM Administrator

Start TSM client

2009-08-17T11:22:00.001+12:00

nohup /usr/tivoli/tsm/client/ba/bin/dsmc sched -quiet > /var/tmp/dsmc_sched.log 2>&1
bg

Setting Default Domain in Win2k3

2009-08-14T15:24:00.002+12:00

Run Regedit and edit the following keys:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon\DefaultDomainName
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon\DefaultUserName
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon\AltDefaultDomainName
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon\AltDefaultUserName

OpenIdP Server rebuilding

2009-06-25T13:29:00.004+12:00

Copy:

users' folders
root's folder
/usr/local/apache-tomcat-5.5.23
/usr/local/shibboleth-idp
/usr/java/jdk1.5.0_11
/var/www/http/*
/etc/httpd/conf/
/etc/httpd/conf.d/shib-idp.conf
/etc/httpd/conf.d/ssl.conf
/etc/httpd/conf.d/redirectToHTTPS.conf

Command:

ln -s /usr/local/apache-tomcat-5.5.23 tomcat
ln -s /usr/java/jdk1.5.0_11 java

Add into /etc/httpd/conf.d/proxy_ajp.conf

ProxyPassReverse /shibboleth-idp/ ajp://localhost:8009/shibboleth-idp/
ProxyPass /shibboleth-idp/ ajp://localhost:8009/shibboleth-idp/

Install LDAP:

compat-openldap.i386
openldap-clients.i386
openldap-servers.i386
openldap-servers-sql.i386

Copy LDAP Database:

slapcat > idp.ldiff # old idp
slapadd -v -l idp.ldiss # new idp
start ldap

Edit shib-idp.conf:

AuthType Basic
AuthName "BeSTGRID Identity Provider"
AuthLDAPBindDN cn=shibboleth,dc=idp,dc=bestgrid,dc=org
AuthLDAPBindPassword "password"
AuthLDAPURL "ldap://idp.bestgrid.org:389/ou=people,dc=idp,dc=bestgrid,dc=org?cn"
AuthBasicProvider ldap
require valid-user
Require ldap-filter objectClass=*
AuthzLDAPAuthoritative on

Start/restart httpd:

service httpd restart

Start Tomcat:

/etc/init.d/tomcat stop
/etc/init.d/tomcat start

Test:

Cron Jobs

/etc/cron.hourly/idp-aaL1-metadata
/etc/cron.hourly/idp-bestgrid-test-metadata

WAYF Test server transfer

2009-06-19T16:03:00.006+12:00

Copy:

users' folders
root's folder
/usr/local/apache-ant-1.7.0
/usr/local/apache-tomcat-5.5.23
/usr/local/shibboleth-idp-metadatatool
/usr/local/shibboleth-wayf
/usr/java/jdk1.5.0_11
/var/www/http/*

Command:

ln -s /usr/local/apache-tomcat-5.5.23 tomcat
ln -s /usr/java/jdk1.5.0_11 java

Add into /etc/httpd/conf.d/proxy_ajp.conf

ProxyPassReverse / ajp://localhost:8009/
ProxyPass / ajp://localhost:8009/

Create self-signed certificate:

openssl req -x509 -nodes -days 1095 -subj '/CN=wayf.test.bestgrid.org/C=NZ/L=Auckland/OU=BeSTGRID/O=Broadband enabled Science and Technology GRID' -newkey rsa:1024 -keyout wayfcert.key -out wayfcert.crt

Copy:

wayfcert.key to /etc/httpd/conf/ssl.key/wayfcert.key
wayfcert.crt to /etc/httpd/conf/ssl.crt/wayfcert.crt

Commands:

ln -s /etc/httpd/conf/ssl.key/wayfcert.key /etc/httpd/conf/ssl.key/server.key
ln -s /etc/httpd/conf/ssl.crt/wayfcert.crt /etc/httpd/conf/ssl.crt/server.crt

Edit /etc/httpd/conf.d/ssl.conf:

SSLCertificateFile /etc/httpd/conf/ssl.crt/server.crt
SSLCertificateKeyFile /etc/httpd/conf/ssl.key/server.key

Start/restart httpd:

service httpd restart

Start Tomcat:

/etc/init.d/tomcat stop
/etc/init.d/tomcat start

Test:

EVO Panda Firewall rules

2009-06-17T10:08:00.000+12:00

# Punch a few holes for proper Panda operation
# Connections from other Panda servers
-A RH-Firewall-1-INPUT -m udp -p udp --dport 46014 -j ACCEPT
# Connections from Koala
-A RH-Firewall-1-INPUT -m tcp -p tcp --dport 46015 -j ACCEPT
-A RH-Firewall-1-INPUT -m udp -p udp --dport 46015 -j ACCEPT
# Connections from Mongoose
-A RH-Firewall-1-INPUT -m udp -p udp --dport 46025 -j ACCEPT
# ApMon from MonALISA -A RH-Firewall-1-INPUT -m udp -p udp --dport 8884 -j ACCEPT
# Connections from any remote H.323/SIP clients
-A RH-Firewall-1-INPUT -m udp -p udp --dport 57000:57400 -j ACCEPT
# Remote management from MonkeySSL
-A RH-Firewall-1-INPUT -m tcp -p tcp --dport 3232 -j ACCEPT
-A RH-Firewall-1-INPUT -m tcp -p tcp --dport 3233 -j ACCEPT
# Remote management from Kiwi
-A RH-Firewall-1-INPUT -m tcp -p tcp --dport 5353 -j ACCEPT
-A RH-Firewall-1-INPUT -m tcp -p tcp --dport 5354 -j ACCEPT

Permission denied for root on NFS volume

2009-04-29T14:05:00.002+12:00

If NFS volume exported with root= option (allow root write access as root) and root on client cannot write a file on mounted NFS volume, flipping R/W permissions for a folder where NFS mounted may solve the problem.

Example: mysql prod slave mounted NFS from datastorage but during starting mysqd there was an error "Permissions denied" (procedure changes ownership of /var/lib/mysql under root). Setting permissions to 777 and back to 755 solved this problem.

TSM notes

2009-02-19T11:34:00.003+13:00

From: Roderic Campbell
Sent: Monday, September 01, 2008 11:35 AM

Your TSM client node DATA.BESTGRID.ORG has been put on a TSM server schedule
to commence backup at 19:00 every night. Is 19:00 a good start time?

In order for the association of the client node with the schedule to become “active”, it is
necessary for the client SysAdmin (you) to stop and restart the TSM client daemon.

ps –ef | grep dsm

should display the TSM client daemon running on your machine. This process can be killed.

Restarting the daemon – somewhere in the system startup suite there will be an entry
similar to:

/usr/tivoli/tsm/client/ba/bin/dsmc sched -quiet >/var/tmp/dsmc_sched.log 2>&1 &

This command can be used ad-hoc to restart TSM daemon.

Before you do this plse let me know if 7pm is OK. – Any changes I make to the schedule will also require
a restart of TSM at your end, so let’s get the schedule finalised first.

Couldn't find device with uuid

2008-06-27T14:15:00.002+12:00

If for most lvm command there are error messages like this:

Couldn't find device with uuid 'RI09tl-vmkB-PLIG-UEOz-v1kB-tsvH-z2DA75'.
Couldn't find all physical volumes for volume group VolGroup00.
Couldn't find device with uuid 'RI09tl-vmkB-PLIG-UEOz-v1kB-tsvH-z2DA75'.
Couldn't find all physical volumes for volume group VolGroup00.

Issue a command:

vgreduce --removemissing VolGroup00

Volume group VolGroup00 becomes accessible and errors are gone. Now it's possible to remove empty volume group by a command

vgremove VolGroup00

Globus - error while copying output files to local machine

2008-05-23T11:33:00.005+12:00

Problem:
Submitting job with copying output files in fileStageOut.

Error:
globusrun-ws: Job failed: Staging error for RSL element fileStageOut
Connection creation error [Caused by: Connection refused]

RSL job description:
(Damn! Blogger doesn't allow to publish XML)

Cause:
Globus uses gsiftp from ng2 under allocated grid user (grid-bestgrid). My local machine doesn't have either host certificate or user certificate for grid-bestgrid user.

Solution:

Apply for a host certificate for my local machine. I don't really need it now. But other users might do need.
Create a script to run a job and add globus-url-copy command to copy files from the gateway to local machine. Files should be deleted on the gateway in the script or in other way. fileStageOut/delete doesn't useful in this case.

Installation PHP with MySQL support

2008-02-29T15:22:00.001+13:00

Download PHP:
http://nz2.php.net/get/php-5.2.5.tar.gz/from/a/mirror

Before run

./configure --with-apxs2=/usr/sbin/apxs --with-mysql

to install PHP be sure that following packages are installed

yum install httpd-devel
yum install libxml2-devel
yum install mysql-devel

To increase a disk size for a VM

2008-02-05T15:56:00.000+13:00

To increase a disk size for a VM:

In XenCenter "Storage" tab add a space
In the VM console create a physical volume: pvcreate /dev/xvdb
Extend a Volume Group for the physical volume: vgextend VolGroup00 /dev/xvdb
Extend a logical volume for the size of new disk: lvextend -L +39G /dev/VolGroup00/LogVol00
Switch to single user mode: init 1
Extend file system to the whole logical volume: resize2fs /dev/VolGroup00/LovVol00

Push logs to log server

2008-01-31T15:14:00.000+13:00

SYSLOG configuration

A line in /etc/syslog.conf to push log files to log server for analyzing.

*.notice;*.alert;kern.err;daemon.err;auth.info @loghost.auckland.ac.nz

PostgreSQL: readline/zlib library not found

2008-01-18T13:55:00.000+13:00

There should be -devel packages of readline/zlib installed in the system.

Xen Administrator Console doesn't connect to XenServer

2007-10-19T15:25:00.000+13:00

If Xen Administrator Console doesn't connect to XenServer or just hanged on than one needs to restart xenagentd daemon:

# service xenagentd restart

iRODS installation

2007-10-17T14:19:00.000+13:00

Run under ordinary user, i.e. irodsmgr;
Create folder for irods and iRodsPostgress and change owner to irodsmgr;
Run install.pl and answer no on first question;
Edit irods/install/install.config:
set
$ADMIN_PW
$RESOURCE_NAME
variables
and uncomment $LOCALHOST="yes";
Run install.pl again.

Requirements for iRods and Postgress:
gcc and dependencies
gcc-c++ and dependencies

Xen P2V conversion

2007-10-17T13:47:00.000+13:00

Shutdown a VM
Power off the VM
Change HVM status to Enable and put XenServer.iso into CDROM
Power on the VM
Follow the conversion procedure
Power off the old VM
Rename the new VM
Change MAC address
Boot first time
Switch off SELinux in /etc/selinux/config
Reboot the VM
Done.

Optimize MySQL table

2007-09-04T10:28:00.000+12:00

OPTIMIZE TABLE should be used if you have deleted a large part of a table or if you have made many changes to a table with variable-length rows (tables that have VARCHAR, VARBINARY, BLOB, or TEXT columns).

For Sakai database the largest table is CONTENT_RESOURCE_BODY_BINARY.MYI
It keeps all files uploaded to Resources Tool. After deleting some files from Resources this table might be fragmented. To optimize its internal structure it's possible to use a mysql command:

optimize table CONTENT_RESOURCE_BODY_BINARY;

or mysql utility:

myisamchk -r --sort-index --analyze \
CONTENT_RESOURCE_BODY_BINARY.MYI

Option -o (--safe-recover) instead of -r should be used if there is an error:

myisamchk: error: sort_buffer_size is to small

AIX: File too large

2007-08-21T10:34:00.000+12:00

It happens when file size large then user's limit. To see limits use a command

# ulimit -a

Change limits:

/etc/security/limits

Networking in XEN

2007-08-06T10:16:00.000+12:00

During last startup of the Gateway server after last Saturday relocation I realized that network interfaces should be UP on boot.

network-bridges custom script:

#!/bin/sh
/etc/xen/scripts/network-bridge.orig $1 netdev=eth0 bridge=xenbr0 vifnum=0
/etc/xen/scripts/network-bridge.orig $1 netdev=eth1 bridge=xenbr1 vifnum=1
/etc/xen/scripts/network-bridge.orig $1 netdev=eth2 bridge=xenbr2 vifnum=2
/etc/xen/scripts/network-bridge.orig $1 netdev=eth3 bridge=xenbr3 vifnum=3
/etc/xen/scripts/network-bridge.orig $1 netdev=eth4 bridge=xenbr4 vifnum=4
/etc/xen/scripts/network-bridge.orig $1 netdev=eth5 bridge=xenbr5 vifnum=5
route add default gw 130.216.189.254

This script calls original network-bridge.orig script from Xen installation. In such sequence all bridges started up correct and all interfaces work well. It is called from xend configuration file during xend loading.

No space left on device /dev/null

2007-08-03T10:28:00.000+12:00

If it's impossible to copy a file with a size more than a particular volume then very likely the reason of that that /dev/null has been overwritten by ordinary file:

# cat 1GB > /dev/null
cat: write error: No space left on device

A solution is to recreate /dev/null as a character file:

> rm -f /dev/null (get rid of the file I had put there)
> mknod /dev/null c 1 3 (some unix magic: char device, major #, minor #)
> chmod a+w /dev/null (permissions)

It works.

BG2 - First Virtual Server

2007-08-01T15:11:00.001+12:00

The specifications for the server are fairly minimal see below:…I suggest 1.5GB of RAM for the Windows 2003 Web/Nesstar server .

We should allow at least 2GB for all future servers.

Web/Nesstar server Hardware/Software Requirements:
Windows XP, 2000 or NT4.
Windows Server 2003
All Service Pack updates should be installed. (If you experience problems with the installation under NT4, check that the video drivers are up to date).
1GB of RAM (more is preferable)
1 GHz CPU or better. A biprocessor machine can be useful, especially during upgrading when the new and the old server might be running side-by-side.
Accessing WebView requires a web browser which is set to accept cookies and is
JavaScript enabled. Security and privacy settings should be set to accommodate this
requirement.
IIS (Internet Information Server) versions 5 or 6:
To install a Nesstar Server on a secondary IP address, please refer to the Nesstar
document on socket pooling.

BeSTGRID Server 2

2007-08-01T13:34:00.000+12:00

To configure and install appropriate software:

~~Download Xen Enterprise~~
~~Acquire a license for the Server~~
~~Acquire Windows Server 2003 distributive~~
~~Apply for IP pool~~
~~Acquire empty CDs~~
~~Burn CDs with Xen images~~
~~Configure RAIDs on BeSTGRID2~~
~~Install Xen Enterprise on BG2~~
~~Install Admin Console on my PC~~
~~Install WinServer2003 on first VM~~
~~Create VMs for WAYF and OpenIdP~~

All done!

Start/Stop Samba

2007-07-30T11:48:00.000+12:00

Now I've added Samba as a subsystem:

# mkssys -s smbd -p /opt/pware/samba/3.0.24/sbin/smbd -a "-D" -u 0

and on default it should start automatically during the system boot. If it isn't started try first:

# startsrc -s smbd

if unsuccessful try next:

# /opt/pware/samba/3.0.24/sbin/smbd -D

Other option is to use files:
/etc/rc.d/rc2.d/Ssmbd
/etc/rc.d/rc2.d/Ksmbd
Now they are in /home/akha103

AVOIDING Disk Crashes

2007-07-30T09:29:00.000+12:00

1. Rule One: don't let this stop your system
* RAID5 or mirror everything
2. Rule Two: monitor error logs
* Make sure you know when a disk failed
3. Rule Three: call hardware support
* That is what they are for
4. Rule Four: Don't meddle
* Only try, if you know what you are doing
5. Rule Five: Read and practise
* Get the Redbooks and try it safely

Formatted list of active Volume groups

2007-07-30T09:22:00.000+12:00

for i in $(lsvg);do lsvg $i;done \
| awk ' BEGIN { printf("%10s\t%10s\t%10s\t%10s\t%10s\n","VG","Total(MB)","Free","USED","Disks") };/VOLUME GROUP:/ \
{ printf("%10s\t", $3) };/TOTAL PP/ { B=index($0,"(") + 1;E=index($0," megaby");D=E-B;printf("%10s\t", substr($0,B,D) );};/FREE PP/ \
{ B=index($0,"(") + 1;E=index($0," megaby");D=E-B;printf("%10s\t", substr($0,B,D) );};/USED PP/ \
{ B=index($0,"(") + 1;E=index($0," megaby");D=E-B;printf("%10s\t", substr($0,B,D) );};/ACTIVE PV/ { printf("%10s\t\n", $3) } '

emails from NGData has root@bestgrid30.math.auckland.ac.nz in From:

2007-07-27T12:12:00.000+12:00

All emails from NGData has root@bestgrid30.math.auckland.ac.nz in From: field.

The reason of that is I created CNAME ngdata.auckland.ac.nz for data.bestgrid.org (bestgrid30.math.auckland.ac.nz) long time ago. And in revers lookup mail server finds bestgrid30 as a sender domain.

I've applied to move ngdata.auckland.ac.nz from bg30 to bg4.

UPD: Fixed on 27.07.2007

Use correct CPU number

2007-07-27T09:54:00.000+12:00

#!/bin/bash
#PBS -u user_name
#PBS -l nodes=1:ppn=8
#PBS -o $PBS_JOBNAME.out
#PBS -e $PBS_JOBNAME.err

#How many procs do I have?
NP=$(wc -l $PBS_NODEFILE | awk '{print $1}')

#cd into the directory where I typed qsub cd $PBS_O_WORKDIR

#run executable
mpiexec -np $NP executable

Pre-install of any APAC packages.

2007-07-26T11:30:00.000+12:00

Pre-install of any APAC packages.
Edit to enable centosrepo:
# nano /etc/yum.repos.d/CentOS-Base.repo

# rpm --import http://mirror.centos.org/centos/4/os/i386/RPM-GPG-KEY-centos4
# cd /etc/yum.repos.d && wget http://www.grid.apac.edu.au/repository/dist/APAC-Grid.repo
# yum install Gpulse Gbuild

Check hostcert.*
openssl x509 -in hostcert.pem -noout -modulus
openssl rsa -in hostkey.pem -noout -modulus

Build NGData
/usr/local/sbin/BuildNgdataVdt161.sh

NGData ToDo

2007-07-25T16:12:00.001+12:00

- set yum repos to VPAC
- install certificate
- install Gbuild
- install NGData

Current Disk Space Allocation on GateWay

2007-07-25T15:53:00.000+12:00

XENHOST

Filesystem Size Used Avail Use% Mounted on
/dev/sda1 9.7G 1.2G 8.0G 14% /
/dev/sda3 56G 5.1G 48G 10% /home

VMHOST

VG #PV #LV #SN Attr VSize VFree
VolumeGroup00 1 29 0 wz--n- 409.84G 326.43G

LV VG Attr LSize
GridSphereR VolumeGroup00 -wi-ao 2.00G
GridSphereS VolumeGroup00 -wi-ao 512.00M
LanguageR VolumeGroup00 -wi-ao 2.00G
LanguageS VolumeGroup00 -wi-ao 512.00M
NG2Root VolumeGroup00 -wi-ao 16.00G
NG2Swap VolumeGroup00 -wi-ao 512.00M
OpenIdpR VolumeGroup00 -wi-ao 2.00G
OpenIdpS VolumeGroup00 -wi-ao 512.00M
SRSData VolumeGroup00 -wi-ao 20.00G
SRSRoot VolumeGroup00 -wi-ao 1.50G
SRSSwap VolumeGroup00 -wi-ao 512.00M
SakaiR VolumeGroup00 -wi-ao 8.00G
SakaiS VolumeGroup00 -wi-ao 512.00M
SakaitR VolumeGroup00 -wi-a- 8.00G
SakaitS VolumeGroup00 -wi-a- 512.00M
ServicesR VolumeGroup00 -wi-ao 4.00G
ServicesS VolumeGroup00 -wi-ao 512.00M
SolverR VolumeGroup00 -wi-ao 1.49G
SolverS VolumeGroup00 -wi-ao 512.00M
SolverU VolumeGroup00 -wi-ao 1.50G
VreR VolumeGroup00 -wi-ao 2.00G
VreS VolumeGroup00 -wi-ao 512.00M
VreU VolumeGroup00 -wi-ao 1.50G
WayfR VolumeGroup00 -wi-ao 2.00G
WayfS VolumeGroup00 -wi-ao 512.00M
WikiProdR VolumeGroup00 -wi-ao 2.93G
WikiProdS VolumeGroup00 -wi-ao 512.00M
WikiR VolumeGroup00 -wi-ao 2.00G
WikiS VolumeGroup00 -wi-ao 512.00M

Auckland is in APAC!!!

2007-07-24T13:17:00.000+12:00

ng2.auckland.ac.nz is in APAC structure now!!!

It is in GOC database: http://goc.grid.apac.edu.au/

AUCKLAND site in APAC MDS/MIP database:
http://www.sapac.edu.au/webmds/webmds?info=indexinfo&xsl=apacgluexsl
Seek 'AUCKLAND' on the page.

Also ng2.auckland.ac.nz is visible in gcc GUI and even it's possible to submit jobs
for MrBayes calculation but they don't run good yet. I'm working on this.

MrBayes Installation

2007-07-19T10:22:00.000+12:00

To provide a service in Computation Grid I decided to install MrBayes.

The home of MrBayes is http://www.mrbayes.net/

The latest version is 3.1.2. In Linux environment this package should be compiled. First attempt to compile was unsuccessful with errors:

bayes.c:45:31: readline/readline.h: No such file or directory

bayes.c:46:30: readline/history.h: No such file or directory

MrBayes manual says:

Depending on which platform/distribution the compilation is being performed on, it may be necessary to install relevent development libraries to enable compilation. Users of Ubuntu 5.10 (Breezy Badger), for instance, may need to install the either the libreadline4-dev or libreadline5-dev package to provide linking during compilation to GNU readline functionality.

The first try to install libreadline5-dev requires to issue 'apt-get install -f' without any package to update system. After this command I could to install libreadline5-dev by:

'apt-get install -f libreadline5-dev'

Then compilation of MrBayes performed well. Now I think I need to install lam-mpi package to make MrBayes mpi compatible. Currently no mpi packages are install on my head node.

Yahoo! Test2 is running!

2007-07-10T16:00:00.001+12:00

Final thing was a connecting between ng2:pbs-logmaker and bestgrid-02:pbs-telltail.
Because of bestgrid-02 is Debian box I had to modify startup file quite heavily. Modified lines are bolded:

#!/bin/sh
# pbs-telltail Starts/stops pbs-telltail daemon.
# Graham Jenkins Nov. 2005. Modified: 20051220
#
# chkconfig: 2345 99 05
# description: pbs-telltail startup script

# Adjust as appropriate
REMOTES="ng2.auckland.ac.nz:2812"
[ -z "$PBS_HOME" ] && PBS_HOME=/opt/torque
[ -z "$TELLTAIL_HOME" ] && TELLTAIL_HOME=/usr/local/pbs-telltail

#. /etc/rc.d/init.d/functions
RETVAL=0
case "$1" in
start ) for Remote in $REMOTES ; do
Host=`echo $Remote | awk -F: '{print $1}'`
Port=`echo $Remote | awk -F: '{print $2}'`
echo -n "Starting pbs-telltail on host: $Host .. port: $Port .. "
$TELLTAIL_HOME/pbs-telltail $PBS_HOME/server_logs $Host $Port
RETVAL=$?; echo; [ $RETVAL -ne 0 ] && break
done
[ $RETVAL -eq 0 ] && touch /var/lock/subsys/pbs-telltail ;;
stop ) echo -n "Shutting down pbs-telltail .. "
killproc pbs-telltail
RETVAL=$?; echo
[ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/pbs-telltail ;;
status) status pbs-telltail
RETVAL=$? ;;
* ) echo "Usage: $0 {start|stop|status}"; exit 1 ;;
esac
exit $RETVAL

stop and status options don't work. I will investigate this later.

I copied /usr/local/pbs-telltail/* from ng2 to bestgrid-02.
Copied pbs-telltail.RH to /etc/init.d/pbs-telltail
And modified as mentioned above. Than started this script.

Before a pbs job had stuck and displayed a message "Job Unsubmitted". After it goes further and display env variables as that specified in test2.rsl file (job description).

Submit PBS job from ng2

2007-07-10T12:49:00.000+12:00

In my configuration there are 3 boxes: ng2, bestgrid-02 (pbs-server), bestgrid-01 (node). Submitting of PBS jobs requires following issues:

All three boxes have to have common folder for user which has been used as a grid user. In my case it's grid-user. I've exported /home/grid-user from bestgrid-02 and mounted it to ng2 and bestgrid-01 into the same folder;
Ideally that should be whole /home folder exported from another box (i.e. data.bestgrid.org) and mounted on all boxes in a chain ng2->pbs-server->nodes;
grid-user has to have passwordless access between all boxes in the chain;
In /home/grid-user/.ssh/known_hosts records for all boxes should be existed for short and long names;
All boxes should be in /etc/hosts of each box.

One way password less SSH

2007-07-06T15:45:00.000+12:00

For user who has to have password less SSH between two hosts, all files in .ssh folder of both hosts must have the same permission attributes:

-rw-r--r-- 1 grid-user grid-user 867 Jul 6 13:51 authorized_keys
-rw------- 1 grid-user grid-user 668 Jun 27 15:27 id_dsa
-rw-r--r-- 1 grid-user grid-user 618 Jun 27 15:27 id_dsa.pub
-rw------- 1 grid-user grid-user 883 Jun 27 15:27 id_rsa
-rw-r--r-- 1 grid-user grid-user 238 Jun 27 15:27 id_rsa.pub
-rw------- 1 grid-user grid-user 554 Jun 27 15:27 identity
-rw-r--r-- 1 grid-user grid-user 358 Jun 27 15:27 identity.pub
-rw-r--r-- 1 grid-user grid-user 918 Jun 27 15:02 known_hosts

In my case I could ssh from ng2 to bestgrid-02 but not back. After Anton's suggestion I found out that on ng2 file authorized_keys had -rw-rw-r-- permissions. After changing to -rw-r--r-- I could ssh from bestgrid to ng2 without password.

qstat: cannot connect to server ... (errno=111)

2007-06-28T15:56:00.000+12:00

I could pass this error only when commented lines

pbs 15000/tcp
pbs_dis 15001/tcp
pbs_dis 15001/udp
pbs_mom 15002/tcp
pbs_mom 15003/udp
pbs_mom 15003/tcp
pbs_sched 15004/tcp

in ng2:auckland.ac.nz:/etc/services file.
I placed those line reading Vladimir's recommendations.

UPD: after some tests I've found out that it's enough to edit first line in this group. It should be pbs_server 15000/tcp and then pbs utilities can connect to pbs_server. Even if on the box where pbs_server is running this line should like that, otherwise pbs utilities don't see the pbs_server.

unknown nfs status return value: -1

2007-06-28T13:11:00.000+12:00

This error message might appear while mounting NFS share exported from AIX box.
That means an importer doesn't have a record in DNS database.

To solve just add an entry about the importer into /etc/hosts file.

Source...

Virtual File System Manager

2007-06-20T16:07:00.000+12:00

Some things to remember while using Virtual File System Manager:

It looks like in Windows environment VFSM requires domain name of remote host should be A record rather than CNAME; If domain name of a host is CNAME, VFSM can't connect to it with an error like this:
In Linux environment VFSM can mount remote file system of a host with both kind of domain names (A record or CNAME);
VFSM can copy files between remote file system and local file system and back and can't copy files between two remote file systems. If user copies files between two remote hosts then files aren't appeared on destination hosts but deleted from source host.
To copy user needs to use shortcuts Ctrl+C and Ctrl+V instead of mouse drag-n-drop.
If there was an attempt to get access to any folder but /tmp as Anonymous user then VFSM didn't give access even to /tmp folder. The best way I know is to restart VFSM.
May be there are some more troubles but I didn't investigate them yet.

CRL has expired

2007-06-19T08:24:00.000+12:00

That means Certificate Revocation List has been expired. This error may appear when create proxy with -verify option:

grid-proxy-init -verify -debug

It's a file in /etc/grid-sequrity/certificates/1e12d831.r0 (for APACGrid CA) Next Update is overdated. Current (for the date of this posting) CRL has inside:

Version 1 (0x0)
Signature Algorithm: md5WithRSAEncryption
Issuer: /C=AU/O=APACGrid/OU=CA/CN=APACGrid/emailAddress=camanager@vpac.org
Last Update: Jun 18 23:50:25 2007 GMT
Next Update: Jul 18 23:50:25 2007 GMT

It hasn't been updated on my bestgrid-02 machine automatically because on unknown reason a line to call fetch-crl.cron script disappeared from crontab file.

To update APACGrid CA CRL use link http://ca.apac.edu.au/pub/crl/cacrl.txt
and place this file as /etc/grid-sequrity/certificates/1e12d831.r0. Also to automatic updates it's a good idea to create cron task for daily or even hourly updates of CRL:

#!/bin/bash

# /opt/vdt/fetch-crl/share/doc/fetch-crl-2.6.2/fetch-crl.cron
# in crontab:
# 12 2 * * * /opt/vdt/fetch-crl/share/doc/fetch-crl-2.6.2/fetch-crl.cron
# Designed to be ran once a day.

if [ -f /etc/sysconfig/fetch-crl ] ; then
. /etc/sysconfig/fetch-crl
fi

CRLDIR=${CRLDIR:-/opt/vdt/globus/TRUSTED_CA/}

QUIET=${QUIET:-yes}

if [ $QUIET = "yes" ] ; then
/opt/vdt/fetch-crl/sbin/fetch-crl --loc $CRLDIR -out $CRLDIR -a 24 --quiet
else
/opt/vdt/fetch-crl/sbin/fetch-crl --loc $CRLDIR -out $CRLDIR
fi

Submitting Job to Run in Canterbury

2007-06-18T12:36:00.000+12:00

To submit job on ng2.canterbury.ac.nz I needed to run ggc (https://www.seegrid.csiro.au/twiki/bin/view/Compsrvices/GenericGridGUI) and fill fields like that.

It's possible to see a new folder with output files using Virtual File System Manager (http://www.vpac.org/twiki/bin/view/Sandbox/UsingVFSManager).

More detailed description is in http://www.bestgrid.org/index.php/Getting_Started_with_BeSTGRIDs_Computational_GRID

Transport e-mails to Sakai

2007-06-14T10:53:00.000+12:00

Sakai has own SMTP server - James. To receive messages in Sakai mailhost.auckland.ac.nz should forward messages to sakai.bestgrid.org. Now MailHost forwards messageg to services.bestgrid.org. It's a unneeded chain ring.
In sakai.properties set parameters for e-mails and use any dns IPs.

ToDo - ask Service Desk to redirect messages for *.bestgrid.org to sakai machine.

Adding user to Samba on AIX

2007-06-08T11:35:00.000+12:00

edit file
/opt/pware/samba/3.0.24/lib/smb.conf

for each user add something like that
# A private directory, usable only by fred. Note that fred requires write
# access to the directory.
;[fredsdir]
; comment = Fred's Service
; path = /usr/somewhere/private
; valid users = fred
; public = no
; writable = yes
; printable = no

Create user account and set password:
/opt/pware/samba/3.0.24/bin/smbpasswd -a userupi

Job submission (globusrun-ws)

2007-06-05T09:53:00.000+12:00

- copy usercert.pem into ~/.globus folder
- copy userkey.pem into ~/.globus folder and change permissions to 600
- run grid-proxy-init -debug
- run globusrun-ws -Ft PBS -submit -S -c /bin/date

ng2 machine should be in acl_hosts list of PBS server (qmgr -c 'l s') and in /etc/hosts.equiv file

Xen error: Link veth4 is missing

2007-06-01T15:05:00.000+12:00

On default Xen has four veths to communicate between bridges and real eths. If one uses a command:

/etc/xen/scripts/network-bridge start netdev=eth4 bridge=xenbr4 vifnum=4

to create bridge xenbr4 and connect it to eth4 then he receives a message with
Link veth4 is missing at the beginning. To fix that situation gateway.auckland.ac.nz:/boot/grup/grub.conf should be changed like this:

kernel /boot/xen-3.gz dom0_mem=262144
module /boot/vmlinuz-2.6.16.33-xen ro root=LABEL=/ netloop.nloopbacks=8
module /boot/initrd.img-2.6.16.33-xen

A number in netloop.nloopbacks parameter is equal to amount of real NICs. Real gateway server shoud be rebooted.

Automate VM creation

2007-06-01T10:05:00.000+12:00

To automate creation of VMs I developed a script based on VPAC recomendations:

#!/bin/sh

#Create a VM:
# allocate logical volumes for root and swap filesystems
# make swap and root filesystems
# untar VM image
# call editor to edit network configuration files
# copy and edit xen template
# create VM
#
#Parameters:
# $1 - Size for root filesystem
# $2 - Name of VM
#
#Author: Andrey Kharuk, 01/06/2007
#

if [ -z "$1" ]; then
echo " " >&2
echo "Usage: `basename $0` disksize(in Mb) VMname" >&2; exit 2
echo " " >&2
echo " " >&2
fi

lvcreate -L $1M -n $2R VolumeGroup00 ;
lvcreate -L 512M -n $2S VolumeGroup00 ;
mkswap /dev/VolumeGroup00/$2S ;
mkfs -t ext3 /dev/VolumeGroup00/$2R ;
mkdir /srv/$2 ;
mount -t ext3 /dev/VolumeGroup00/$2R /srv/$2 ;
cd /srv/$2 ;
tar xjpf /etc/xen/vm-image-0.0.1.tar.bz2 ;
nano /srv/$2/etc/sysconfig/network ;
nano /srv/$2/etc/sysconfig/network-scripts/ifcfg-eth0 ;
cp /etc/xen/template /etc/xen/$2 ;
nano /etc/xen/$2 ;
cd ..
umount /srv/$2 ;

xm create -c $2

Tar-ball 'vm-image-0.0.1.tar.bz2' is an image of fresh installed CentOS 4.0 on VM.
Both files reside in gateway.bestgrid.org:/etc/xen

globus-ws is on now (ng2.auckland.ac.nz)

2007-05-21T16:04:00.000+12:00

To install hostcert I used a procedure from:
http://www.vpac.org/twiki/bin/view/APACgrid/HostCertRequestAPAC

globus-ws requires /etc/grid-cequirity/containercert.pem
I've created copies:
cp /etc/grid-cequirity/hostcert.pem /etc/grid-cequirity/containercert.pem
cp /etc/grid-cequirity/hostcert_request.pem /etc/grid-cequirity/containercert_request.pem
cp /etc/grid-cequirity/hostkey.pem /etc/grid-cequirity/containerkey.pem

Owner of those files is daemon:

-rw-r--r-- 1 daemon daemon 2630 May 21 00:02 containercert.pem
-rw-r--r-- 1 daemon daemon 1770 May 21 00:03 containercert_request.pem
-r-------- 1 daemon daemon 1679 May 21 00:03 containerkey.pem

To check the certificate use a command:

openssl verify -CApath /etc/grid-security/certificates /etc/grid-security/containercert.pem

services.bestgrid.org

2007-05-15T15:54:00.000+12:00

Established services on this host are:

rsync
mail server (under construction)

Planned services are:

Subversion server

tar: Removing leading '/' from member names

2007-05-15T11:30:00.000+12:00

To eliminate this message while creating an archive file with tar use the following syntax:

> tar cj -C / var/www > backup.tar.bz2

instead of

> tar cjf backup.tar.bz2 /var/www

BeSTGrid IP pool

2007-05-11T13:28:00.000+12:00

IP Addresses Pool for BeSTGrid computer farm

Current Disk Space Allocation on GateWay

2007-05-11T13:11:00.000+12:00

XENHOST

Filesystem Size Used Avail Use% Mounted on
/dev/sda1 9.7G 965M 8.2G 11% /
/dev/sda3 56G 4.3G 49G 8% /home

VMHOST

VG #PV #LV #SN Attr VSize VFree
VolumeGroup00 1 17 0 wz--n- 409.84G 342.86G

LV VG Attr LSize
GridSphereR VolumeGroup00 -wi-ao 2.00G
GridSphereS VolumeGroup00 -wi-ao 512.00M
LanguageR VolumeGroup00 -wi-ao 2.00G
LanguageS VolumeGroup00 -wi-ao 512.00M
NG2Root VolumeGroup00 -wi-ao 16.00G
NG2Swap VolumeGroup00 -wi-ao 512.00M
SRSData VolumeGroup00 -wi-ao 20.00G
SRSRoot VolumeGroup00 -wi-ao 1.50G
SRSSwap VolumeGroup00 -wi-ao 512.00M
SakaiR VolumeGroup00 -wi-ao 8.00G
SakaiS VolumeGroup00 -wi-ao 512.00M
SakaitR VolumeGroup00 -wi-a- 8.00G
SakaitS VolumeGroup00 -wi-a- 512.00M
ServicesR VolumeGroup00 -wi-ao 4.00G
ServicesS VolumeGroup00 -wi-ao 512.00M
SolverR VolumeGroup00 -wi-ao 1.49G
SolverS VolumeGroup00 -wi-ao 512.00M

Xen's VMs and IPs

2007-05-11T12:16:00.000+12:00

In case to provide separate physical network interfaces for each BestGrid VM we need two IPs for each VM. One IP is for a bridge between NIC and VM and another one is for VM itself.

So to provide low IPs (189.2,3,4,5,6) for BeSTGrid GateWay VMs I need to swap NIC IPs to 189.38-42 range.

Unfortunately I have to state that our IP pool is over now. There are only two spare IPs - 189.37 and 189.43.

I'm going to start swap today after lunch.

Is it better than paper and simple notebook?

2007-05-10T14:40:00.000+12:00

I try to follow my colleague Anton Gerdelan to keep a blog about my problems and achievements instead of to write then into a paper notebook. May be it's better... The bad thing is I can't keep here passwords for my stuff :)

But it's possible to keep it private... Hm... Anyway that's unsecured way to keep passwords here even in a private blog.