Network Ports and Firewalls.

New Approach as of March 2007

It has been determined that all sites that wish to be part of the APACGrid must have a number of predetermined ports open to the whole world. Please add any suggested entries and if appropriate, put some discussion and/or your name in the notes column.

Ashley: There is a list of GatewayIps#Globus_Ports further down the page. Do we need to duplicate it here?

• This list once comments are all cleaned up - should replace any others? In fact site IP addresses are no longer needed if this is agreed on - makes for much simpler firewall configuration. (Daniel)

Concerns:

• GFTP, Webdav, GSISSH - risk of very large data transfers that are Off-Net. No way to monitor how much transferred, could be very expensive. (Daniel)

Notes:

• This list assumes that all of these ports are allowed outgoing by all sites. Or allowed via Proxy (80, 443). Globus-ws does not work via proxy? so port must be allowed direct. (Daniel)
• Everything will work behind NAT, except active GFTP (which is also streaming -s option with submit). See: FirewallAndStaging (Daniel)

Application	Port(s) IN	Notes	Added by
GFTP	2811	tcp+udp	Ashley
	2811	udp not required!	Daniel
Globus-WS	8080 & 8443	tcp	Ashley
	8080	why do we need this port?	Youzhen
GLOBUS_TCP_PORT_RANGE	40000:41000	tcp	Ashley
		Removed comment (Notes above) 'In+Out (This needs to be world for GridFTP to work)'	Daniel
WebDav			David
	probably 443 and/or 8443		Daniel
GSISSH	22		David
		optional? and ngdata only? some may still restrict by hosts.allow	Daniel
HTTP	80	Portal	Youzhen
HTTPS	443	Portal,GUMS for authtool	Youzhen
		443 redirected by some sites? We all should? Using 8443 to run as non-root user only?	Daniel
HTTPS	8443	Portal,GUMS for authtool	Youzhen
VOMRS	8443		David
		VPAC only - but Grix uses it so you'll need outgoing ports (shouldn't be a problem)	Daniel
Voms	15000,15001,...	depending on how many VOs are hosted on a voms server	Markus
		VPAC only - is it used by GRIX and therefore outgoing ports required? Yes.(Markus)	Daniel
MDS	8443	QUT, SAPAC - specific sites only, hosting a replica	Daniel
Myproxy	7512	myproxy1, myproxy2 - specific sites only QUT, AC3?	Daniel (Vladimir)
CA	80, 443, 8080	APAC only? other CAs will use these ports and need to be contacted for CRL (wgetrc for proxy on ng2, ... works)	Daniel (Vladimir)

A further range of ports will need to be opened to other APACGrid Sites

Application	Port(s)	Notes

Site Addresses

TPAC gateway:

Subnet: 202.6.77.32/28 (No machines other than APAC Grid Gateway hosts will be located in this subnet)

nggateway.sf.utas.edu.au	202.6.77.34
ngdev.sf.utas.edu.au	202.6.77.35
ngportal.sf.utas.edu.au	202.6.77.36
ng2.sf.utas.edu.au	202.6.77.37

-- JohnDalton - 08 Nov 2005

QPSF /QUT gateway:

Subnet: 131.181.86.96/28

131.181.86.97   gridgateway.its.tils.qut.edu.au
131.181.86.98   ng1.qut.edu.au
131.181.86.99   ng2.qut.edu.au
131.181.86.100  ngportal.qut.edu.au
131.181.86.101  ngdata.qut.edu.au
131.181.86.102  myproxy.qut.edu.au

-- AshleyWright - 08 Nov 2005

QPSF /UQ gateway

nggateway.hpcu.uq.edu.au   130.102.163.140
ng1.hpcu.uq.edu.au         130.102.163.141
ng2.hpcu.uq.edu.au         130.102.163.142
ngportal.hpcu.uq.edu.au    130.102.163.143
ngdata.hpcu.uq.edu.au      130.102.163.144
ng2.esscc.uq.edu.au        130.102.167.121
ngportal.esscc.uq.edu.au   130.102.167.122

QPSF node now has final IP addresses

-- WillHsu - 05 Oct 2006

ANU gateway

ngdom0.apac.edu.au       192.43.239.20
ng1.apac.edu.au          192.43.239.21
ng2.apac.edu.au          192.43.239.22
ngdata.apac.edu.au       192.43.239.23
ngportal.apac.edu.au     192.43.239.24
ngportaldev.apac.edu.au  192.43.239.25
nglcg.apac.edu.au        192.43.239.26
ngdev.apac.edu.au        192.43.239.27

-- StephenMcMahon - 15 Nov 2005

iVEC

nggateway.ivec.org       192.65.130.180
ng1.ivec.org             192.65.130.181
ng2.ivec.org             192.65.130.182
ngportal.ivec.org        192.65.130.183
ngdata.ivec.org          192.65.130.184

-- DarranCarey - 15 Nov 2005

VPAC

ng1.vpac.org               131.170.184.173 => 202.158.218.201
ng2.vpac.org               131.170.184.174 => 202.158.218.202
ngdata.vpac.org            131.170.184.177 => 202.158.218.204
ngportal.vpac.org          131.170.184.175 => 202.158.218.203
ca.apac.edu.au             131.170.184.70 => 202.158.218.200
vomrs.vpac.org             131.170.184.32 =>  202.158.218.210    
myproxy2.apac.edu.au       131.170.184.167 => 202.158.218.205
edda.vpac.org              131.170.184.61 => 202.158.218.36 (For job submission)
grid.vpac.org              131.170.184.166 => 202.158.218.208 (ditto)

New addresses shown above are expected to become effective during January 2007.

-- GrahamJenkins - 20 Oct 2006

UniMelb:

charm.hpc.unimelb.edu.au   128.250.7.90
possum.unimelb.edu.au      128.250.150.101
emu.unimelb.edu.au         128.250.150.103
dingo.unimelb.edu.au       128.250.7.96
alfred.hpc.unimelb.edu.au  128.250.7.120 (For job submission.)
aulcg-rb.ph.unimelb.edu.au 128.250.50.236 (For job submission.)

-- LyleWinton - 05 Sep 2006

JCU gateway:

ng0.hpc.jcu.edu.au       137.219.6.34
ng1.hpc.jcu.edu.au       137.219.6.35
ng2.hpc.jcu.edu.au       137.219.6.36
nglcg.hpc.jcu.edu.au     137.219.6.37
ngdata.hpc.jcu.edu.au    137.219.6.38
ngportal.hpc.jcu.edu.au  137.219.6.39
ngdev.hpc.jcu.edu.au     137.219.6.40

SAPAC

grid.sapac.edu.au           129.127.96.84 (for job submission)
ng1.sapac.edu.au           129.127.96.101
ng2.sapac.edu.au           129.127.96.102
ngportal.sapac.edu.au     129.127.96.103
ngdata.sapac.edu.au       129.127.96.104
ng2dev.sapac.edu.au       129.127.96.105
ng1dev.sapac.edu.au       129.127.96.106
ngportaltest.sapac.edu.au       129.127.96.107
mds.sapac.edu.au           129.127.96.100

ac3

nggateway.ac3.edu.au     203.202.30.65
ng1.ac3.edu.au           203.202.30.66
ng2.ac3.edu.au           203.202.30.67
ngdata.ac3.edu.au        203.202.30.68
ngportal.ac3.edu.au      203.202.30.69
nglcg.ac3.edu.au         203.202.30.70
ngdev.ac3.edu.au         203.202.30.71
ngportaldev.ac3.edu.au   203.202.30.72

CSIRO

apacgridgw.hpsc.csiro.au    150.229.76.10  -->  150.229.74.138
ng2.hpsc.csiro.au                150.229.76.12  -->  150.229.74.140
ng2dev.hpsc.csiro.au          150.229.76.8  -->  150.229.74.141
ngdata.hpsc.csiro.au          150.229.76.14  -->  150.229.74.143
ngportal.hpsc.csiro.a          150.229.76.13  -->  150.229.74.142
nggums.hpsc.csiro.            150.229.76.7  -->  150.229.74.144

These changes will occur Saturday 2007-05-19 8:00 - 10:00 AEST

Proposed Subnets for above IPs

192.43.239.0/255.255.255.0      ANU
150.229.76.0/255.255.255.240   CSIRO  --> 150.229.74.128/255.255.255.192
130.116.144.1/255.255.252.0     CSIRO
203.202.17.96/255.255.255.240   AC3 Existing HPC network
203.202.30.64/255.255.255.192   AC3 New HPC network
130.102.163.0/255.255.255.0     UQ
130.102.167.0/255.255.255.0     UQ-ESSCC
129.127.96.64/255.255.255.192   SAPAC
202.6.77.32/255.255.255.240     TPAC
131.170.184.0/255.255.255.0  
192.65.130.0/255.255.255.0      iVEC
131.181.240.0/255.255.255.0     QUT
131.181.86.96/255.255.255.240   QUT 
131.170.184.0/255.255.255.0     VPAC
128.250.50.236                  UniMelb
128.250.7.64/255.255.255.192    UniMelb
128.250.150.101/255.255.255.64  UniMelb
137.219.15.0/255.255.255.0      JCU
137.219.6.33/255.255.255.240    JCU

-- WillHsu - 05 Oct 2006

Firewall settings

For sites that don't have a firewall on their network iptables can be configured on the gateway machine to restrict access to it. Here's how it was done at ANU. It also opens ports defined by GLOBUS_TCP_PORT_RANGE which should be set for all services running on VMs on this gateway.

Globus Ports

Service	Type	Ports	Direction	Needed on
globus-gatekeeper	TCP	2119	In+Out	Ng1, Ng2, NgPortal? (Out), ngdata
gsiftp	TCP+UDP	2811	In+Out	Ng1, Ng2, NgPortal? (Out)
MDS	TCP	2135	In+Out	Ng1
globus-ws	TCP	8443	In+Out	Ng2, NgPortal? (Out)
Apache	TCP	80, 443	In	NgPortal?
Tomcat	TCP	8080, 8443	In	NgPortal?
Tomcat	TCP	8443	In	NgGums?
MyProxy	TCP	7512	Out	Ng1, Ng2, NgPortal?
GLOBUS_TCP_PORT_RANGE	TCP	40000:41000	In+Out	All
SRB	TCP	5545
SRB	TCP	20000:20200

-- AshleyWright - 20 Jan 2006

Enable appropriate kernel modules

cd /usr/local/src/xen-2.0/linux-2.6.11-xen0/
make ARCH=xen menuconfig

The following config flags were set ...

IP_NF_TARGET_REJECT=m
IP_NF_MATCH_MULTIPORT=m
IP_NF_MATCH_STATE=m
IP_NF_MATCH_CONNTRACK=m
IP_NF_CONNTRACK=m
NETFILTER=y
IP_NF_FILTER=m

Here's a copy of the resulting config file.

• config-2.6.11.12-xen0: xen0 kernel config file including iptables modules

To use it copy it to /usr/local/src/xen-2.0/linux-2.6.11-xen0/.config and do

make ARCH=xen oldconfig
make ARCH=xen modules
make ARCH=xen modules_install

You shouldn't need to reinstall the kernel itself or even reboot.

Configure iptables

The file /etc/sysconfig/iptables can be edited to configure iptables. Here's a copy of the file used on the ANU gateway.

• iptables: iptables configuration file for ANU gateway

Put it in place and edit it where appropriate. Note that it allows all traffic from a couple of local networks and defines a list of networks or hosts that grid connections will work for.

Make sure that iptables is on with

chkconfig iptables on
service iptables restart

-- GrahamJenkins - 04 Jan 2007