Securing Your Slice – Part 1

This post is intended to outline the fundamental security precautions for the virtual and baremetal servers that are launched on ExoGENI testbed.

ExoGENI provides experimenters the ability to use their own images when creating slices and launching virtual machines. Testbed infrastructure is well-isolated from the slivers, providing a lot of flexibility to the experimenters on virtual and baremetal servers. At the same time, since individual slivers are created with interfaces on hosting campus IP networks, each virtual or baremetal server should be carefully administered during or after slice creation to ensure proper security measures are taken.

Security is a highly complicated area of concern in server administration. However, we want to outline the fundamental measures for protection of the virtual or baremetal servers which have public internet access. These measures should be added on top of the default OS installation to restrict and control access to the servers, minimize the risk of being a vulnerable hot spot within the campus network.

In this post, we will describe minimally necessary security measures for a linux server regardless of the flavor (CentOS, Ubuntu, Debian etc.), and provide example scripts/commands which can be used during slice creation. (These scripts/commands are based on CentOS 6 distribution and equivalent commands need to be gathered for Ubuntu, Debian, etc.)

“Post Boot Scripts” feature of ORCA can be used to inject security configuration to the nodes or node groups. Details about Post Boot Scripts and templates on this page can be considered as a valuable resource for configuring the virtual or baremetal servers as well as rich scripting and templating capabilities of ExoGENI.

 

1. Servers should include up-to-date packages

After the VM is created, packages can be updated by using a portion of postboot script below:

yum -y update

For kernel updates, rebooting the server is needed. However, VM nodes on ExoGENI testbed are not safely rebootable. Because of some complexities introduced by the virtualization infrastructure, and udev schemes, connectivity with the virtual machines cannot be ensured after rebooting. We do not suggest rebooting the servers on ExoGENI testbed until we implement network device. This will be explained in the following posts. Rebooting may still be needed for system library updates such as glibc. If there is a security concern and an update is needed for such packages, then updating the image, saving it and using that image for slice creation may be necessary.

It is a best practice to upload updated image files (kernel, ramdisk, filesystem) to the web-server and boot up the virtual machines by using the up-to-date images.

 

2. User Authentication

SSH public keys are injected into the virtual or baremetal servers during slice creation. Besides, password authentication for remote root login through SSH should be disallowed by editing the line shown below in sshd_config file:

PermitRootLogin without-password

sshd_config file can be updated by using a portion of postboot script below:

sed -i 's/\#PermitRootLogin yes/PermitRootLogin without-password/g' /etc/ssh/sshd_config
service sshd reload

 

3. Firewall configuration

Inbound and outbound traffic can be controlled by a firewall such as iptables.

Iptables utilizes built-in tables (mangle, filter, nat) to process packets. Each table has a group of chains that represent the actions to be performed on the packet. Built-in chains in filter table are INPUT, OUTPUT and FORWARD. Rules are added to the chains. A packet is checked against each rule in turn (from top to the bottom). An action is taken to ACCEPT or DROP the packet that matches the rule. No further processing is done after a matching rule is found and packet is processed (order of the rules is significant). If a packet passes down through all of the rules in the chain and reaches the bottom without being matched against any rule, then the default policy for that chain is applied to the packet.

Hardening a server with iptables should be taken seriously. IP addresses or networks (both source and destination) as well as ports should be specified to allow traffic for the required connections and reject all other traffic. Default firewall policy and some kernel parameters need to be adjusted, too. Although many useful resources about hardening servers and firewall configuration can be found on the internet, we will prepare a dedicated post to elaborate on this topic for a baseline configuration that can be used for most of the slices. This page can be used as a good starting point to learn about iptables.

Below, we provide a basic set of rules that allows all outgoing connections and blocks all unwanted incoming connections:

# Flush all existing rules
iptables –F

# Set default policy on each chain
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT

# Accept incoming packets destined for localhost interface
iptables -A INPUT -i lo -j ACCEPT

# Accept incoming packets that are part of/related to an already established connection
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Accept incoming packets for SSH connections
iptables -A INPUT -p tcp --dport 22 -j ACCEPT

# Accept incoming packets that belong to icmp protocol
iptables -A INPUT -p icmp -j ACCEPT

If no firewall rules are present, then a basic set of rules can be created and activated using a portion of postboot script below:

iptables –F
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -p icmp -j ACCEPT
service iptables save

Firewall rules can be checked as below:

-bash-4.1# iptables -nvL
Chain INPUT (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           
   38  4071 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:22 
    0     0 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 24 packets, 2254 bytes)
 pkts bytes target     prot opt in     out     source               destination     

 

4. Network services access control

Controlling access to network services can be an important task to provide a balance between flexibility and security within the research-oriented test environment. TCP wrappers is a mechanism used to allow or deny hosts access to the network services. Access files (/etc/hosts.allow and /etc/hosts.deny) are used to determine whether a client is allowed or not. Details on this page can be considered as a valuable resource for configuration.

One common use-case is to restrict access to the portmap service when NFS is being utilized within the slice. Since NFS relies on portmap service which is a dynamic port assignment daemon for RPC services, information about the running services is revealed and can be obtained by an “rpcinfo” request from the servers.

It is critical to restrict access to portmap service through the public interface, and allow access only from the data-plane network. Also, data-plane IP addresses and not public IP addresses should be used in /etc/hosts, /etc/exports, /etc/fstab and other relevant files for NFS configuration.

Steps below should be taken to configure an NFS server:

– On the NFS server host, add the line below to /etc/hosts.deny to allow “rpcinfo” queries only from the data-plane network

rpcbind: ALL EXCEPT <DATAPLANE NETWORK>

Example:

rpcbind: ALL EXCEPT 172.16.0.0/255.255.0.0

– On NFS clients, there is no need to run rpcbind service, and it can be disabled.

Network service access rule can be added by using a portion of postboot script below (Note that data-plane network should be replaced with the network within the slice):

cat << EOF >> /etc/hosts.deny
rpcbind: ALL EXCEPT 172.16.0.0/255.255.0.0
EOF

– Check RPC information that NFS server reveals from the client

# No RPC information is returned for the query from the public interface of the NFS server (192.1.242.62)

bash-4.1# rpcinfo -p 192.1.242.62
rpcinfo: can't contact portmapper: RPC: Authentication error; why = Client credential too weak

# RPC information is returned for the query from the data-plane interface of the NFS server (172.16.0.5)

-bash-4.1# rpcinfo -p 172.16.0.5
   program vers proto   port  service
    100000    4   tcp    111  portmapper
    100005    1   udp  54381  mountd
    100003    3   tcp   2049  nfs
    100227    3   udp   2049  nfs_acl
    100021    1   udp  45462  nlockmgr
    (Sample output shown. Some output omitted)

It is evident that all ExoGENI Testbed users are well aware of the importance of security. A major concern is that security-related problems trigger some restrictions and degradations on the infrastructure. Proposed measures should be applied to every slice as a primary task. Taking precautions to secure your slices will accommodate a rather secure environment both for your precious data/work and the rest of the world.

 

Appendix:

#!/bin/bash

sed -i 's/\#PermitRootLogin yes/PermitRootLogin without-password/g' /etc/ssh/sshd_config
service sshd reload

iptables –F
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -p icmp -j ACCEPT
service iptables save

cat << EOF >> /etc/hosts.deny
rpcbind: ALL EXCEPT 172.16.0.0/255.255.0.0
EOF

yum -y update

 

Have something to add?

Loading Facebook Comments ...