Network Isolation for Bare Metal

By default, bare metal nodes on each Chameleon site share the same local network (shared VLAN and IP subnet). However, some experiments may require more network isolation, which is now supported by Chameleon.

Our implementation of network isolation is based on dynamically managed VLANs (network layer 2) associated with user-configured private IP subnets (network layer 3). This means that all network communications local to the IP subnet or the broadcast domain (such as Ethernet broadcast, ARP, IP broadcast, DHCP, etc.) will be restricted to the user-configured network and its associated VLAN. This feature enables a range of experiments in networking and security. For example, this allows running your own DHCP server to configure virtual machines running on bare metal nodes, without impacting other users.

Please note the following:

To use this feature, you will need to create a dedicated network and router.  You can use a heat template or use the OpenStack dashboard. The procedures are explained below.

Create a Network with Heat Template

  1. From panel Orchestration, choose Stacks:

  2. Click Launch Stack to start using a heat template:

  3. For Template Source, select URL:

  4. In the Template URL field, copy-and-paste in the URL: https://raw.githubusercontent.com/ChameleonCloud/heat-templates/master/network-isolation/network-isolation.yaml

  5. Press Next to go to the Launch Stack screen

  6. At the Launch Stack screen, choose a simple name

  7. Your password is required to perform certain operations; please enter it in the box

  8. Set a private IP range that does not overlap with another; a good rule is to use the last 2 or 4 digits of your project #:

    For example, my project # is 817790:

    1. For a unique 10.xx.yy.0/24 address range, I would use 10.77.90.0/24

    2. For a unique 172.16-31.x.0/24 or 192.168.x.0/24 address range, I would use 172.16.90.0/24 or 192.168.90.0/24

    3. Numbers 100-254 are not used by basic application of this rule and therefore can be used whenever there are conflicts (i.e. in the rare case where the last 2-4 digits of your project is the same as another project, and hence your desired IP subnet range is already in use).

  9. The first IP adddress in the DHCP range should never be *.1, which will be used by the router.  Nor should the first address be *.2 for reasons which will be explained in step 11

  10. The last IP address in the range must be less than *.255

  11. Start creating the network and routers with “Launch”

  12. Congratulations!  Your network and router have been created

    Click on the newly created “stack” to see its details

Create a Network with OpenStack Dashboard

  1. From panel Network, choose Networks:


     

  2. Click “Create Network”


     

  3. Name the network:


     

  4. Make sure “Create Subnet” is checked:


     

  5. Click Next:


     

  6. Name the subnet:


     

  7. Set a Network Address that does not overlap with another subnet; a good rule is to use the last 2 or 4 digits of your project #:



    For example, my project # is 817201:

    1. For a unique 10.x.x.0/24 address range, use 10.72.01.0/24

    2. For a unique 172.16-31.x.0/24 or 192.168.x.0/24 address range, use 172.16.1.0/24 or 192.168.1.0/24
       

    3. Numbers 100-254 are not used by basic application of this rule and therefore can be used whenever there are conflicts (i.e. in the rare case where the last 2-4 digits of your project is the same as another project, and hence your desired subnet range is already in use).
       

  8. Set a Gateway (or leave blank to use the default):

  9. Click Next:


     

  10. Make sure DHCP is enabled:


     

  11. Specify DHCP allocation Pool(s):


    Define the allocation pools to be within the network address allocated to the subnet. It cannot contain the IP allocated to the gateway.

     

  12. Specify DNS Name Servers:

    For Chameleon UC, the DNS name servers are:

    • 130.202.101.6

    • 130.202.101.37


       

  13. Click Create:


     

  14. Check to see the network is created without errors:


     

  15. If you see an error like this:



    Pick a different subnet range (see step 11 e.g. use 10.72.101.0/24 instead)

 

Create a router with OpenStack Dashboard

 

  1. Click on Routers


     

  2. Under Routers lists, click Create Router:


     

  3. Name the router:


     

  4. Select “public” as the External Network if you want to have external access:


     

  5. Click “Create Router”:


     

  6. Now to connect this router to your network, click on the router's name:


     

  7. Under Router Details, click on Interfaces:


     

  8. Click Add Interface:


     

  9. Select the network and subnet you created:


     

  10. Click Add Interface button:


     

  11. Noticed that it has automatically picked the gateway IP you assigned to your subnet:

Use the new network when launching instances

  1. When launching a new instance, under the Networking tab


     
  2. There now will be new options: you will see isolated networks created under your project, as well as the default shared network (named sharednet1).


     

  3. Select the network your instance will be using. If you want to use network isolation, select one of the isolated network created under your project.


     

  4. Launch the instance!

Delete the network and router with Heat Template

  1. To delete the network and router, go to Orchestration -> Stacks, and select your stack and use the “Delete Stacks” button or use the drop-down Actions menu:

  2. Confirm the deletion:

  3. It will take a few seconds to finish the deletion

Delete the network and router with OpenStack Dashboard

  1. First make sure all instances using them are terminated

  2. Click on Routers:


     

  3. Click on the name of your router to see its details, and select Static Routes:


     

  4. Click Delete Static Route:


     

  5. A confirmation appears, confirm to Delete Static route:


     

  6. Go to the Interfaces tab:


     

  7. Delete gateway interface:


     

  8. Confirm Delete Interface:


     

  9. Now the router can be safely deleted:

  10. Confirm Delete Router:


     

  11. Verify that the router is deleted:

  12. Now go delete the network:


     

  13. Use the drop down menu:



    Or check the checkbox and then use the Delete Network button
     

  14. Confirm Delete Network:

Using Openstack CLI to create/delete network and router

### Create Network
 
#   Create Network for a Tenant
#   Select Tenant Network
#   Network will be created with an available segmentation ID within the tenant VLAN pool
 
PHYSICAL_NETWORK_TENANT="physnet1"
NET_TYPE="vlan"
NET_NAME="${PHYSICAL_NETWORK_TENANT}-net-1"
 
openstack network create --provider-network-type ${NET_TYPE} \
                         --provider-physical-network ${PHYSICAL_NETWORK_TENANT} \
                         ${NET_NAME}
 
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | UP                                   |
| availability_zone_hints   |                                      |
| availability_zones        |                                      |
| created_at                | 2018-01-03T21:29:39Z                 |
| description               |                                      |
| dns_domain                | None                                 |
| id                        | b60b81a8-1848-483f-b63b-b99dc1d2e384 |
| ipv4_address_scope        | None                                 |
| ipv6_address_scope        | None                                 |
| is_default                | None                                 |
| is_vlan_transparent       | None                                 |
| mtu                       | 1500                                 |
| name                      | physnet1-net-1                       |
| port_security_enabled     | False                                |
| project_id                | 63700f885ef14a8a8baf7766e275b189     |
| provider:network_type     | vlan                                 |
| provider:physical_network | physnet1                             |
| provider:segmentation_id  | 3026                                 |
| qos_policy_id             | None                                 |
| revision_number           | 2                                    |
| router:external           | Internal                             |
| segments                  | None                                 |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   |                                      |
| tags                      |                                      |
| updated_at                | 2018-01-03T21:29:39Z                 |
+---------------------------+--------------------------------------+
 
Network is created with segmentation ID 3026
(This is the VLAN tag that will be used for stitching).
 
 
### Create Subnet
 
PHYSICAL_NETWORK_TENANT="physnet1"
NET_TYPE="vlan"
NET_NAME="${PHYSICAL_NETWORK_TENANT}-net-1"
 
SUBNET_NAME="subnet-${NET_NAME}"
DNS_NAMESERVER1="130.202.101.6"
DNS_NAMESERVER2="130.202.101.37"
 
# User-defined CIDR, gateway and DHCP allocation pools for the subnet
CIDR="172.16.100.0/24"
GATEWAY_IP="172.16.100.1"
DHCP_ALLOCATION_START="172.16.100.101"
DHCP_ALLOCATION_END="172.16.100.200"
 
openstack subnet create --subnet-range ${CIDR} \
                   --dhcp \
                   --allocation-pool start=${DHCP_ALLOCATION_START},end=${DHCP_ALLOCATION_END} \
                   --dns-nameserver ${DNS_NAMESERVER1} \
                   --dns-nameserver ${DNS_NAMESERVER2} \
                   --gateway ${GATEWAY_IP} \
                   --network ${NET_NAME}   \
                   ${SUBNET_NAME}
 
+-------------------------+--------------------------------------+
| Field                   | Value                                |
+-------------------------+--------------------------------------+
| allocation_pools        | 172.16.100.101-172.16.100.200        |
| cidr                    | 172.16.100.0/24                      |
| created_at              | 2018-01-03T21:35:42Z                 |
| description             |                                      |
| dns_nameservers         | 130.202.101.37, 130.202.101.6        |
| enable_dhcp             | True                                 |
| gateway_ip              | 172.16.100.1                         |
| host_routes             |                                      |
| id                      | 0f83f130-43e3-4193-828f-2c69c8e64603 |
| ip_version              | 4                                    |
| ipv6_address_mode       | None                                 |
| ipv6_ra_mode            | None                                 |
| name                    | subnet-physnet1-net-1                |
| network_id              | b60b81a8-1848-483f-b63b-b99dc1d2e384 |
| project_id              | 63700f885ef14a8a8baf7766e275b189     |
| revision_number         | 2                                    |
| segment_id              | None                                 |
| service_types           |                                      |
| subnetpool_id           | None                                 |
| tags                    |                                      |
| updated_at              | 2018-01-03T21:35:42Z                 |
| use_default_subnet_pool | None                                 |
+-------------------------+--------------------------------------+
 
 
### Create Router, add interfaces and routes
 
ROUTER_NAME="router-${NET_NAME}"
EXTERNAL_NET="public"
NEXT_HOP="172.16.100.100"
DESTINATION="10.140.80.0/22"
 
openstack router create ${ROUTER_NAME}
 
+-------------------------+--------------------------------------+
| Field                   | Value                                |
+-------------------------+--------------------------------------+
| admin_state_up          | UP                                   |
| availability_zone_hints |                                      |
| availability_zones      |                                      |
| created_at              | 2018-01-03T21:36:11Z                 |
| description             |                                      |
| distributed             | False                                |
| external_gateway_info   | None                                 |
| flavor_id               | None                                 |
| ha                      | False                                |
| id                      | 8a9fef13-416b-4995-ba1c-cca588592862 |
| name                    | router-physnet1-net-1                |
| project_id              | 63700f885ef14a8a8baf7766e275b189     |
| revision_number         | None                                 |
| routes                  |                                      |
| status                  | ACTIVE                               |
| tags                    |                                      |
| updated_at              | 2018-01-03T21:36:11Z                 |
+-------------------------+--------------------------------------+
 
openstack router add subnet ${ROUTER_NAME} ${SUBNET_NAME}
openstack router set --external-gateway ${EXTERNAL_NET} \
                    --route destination=${DESTINATION},gateway=${NEXT_HOP} \
                    ${ROUTER_NAME}
                   
 
### Check router configuration
                   
openstack router show ${ROUTER_NAME}
 
+-------------------------+--------------------------------------------------------------------------------------+
| Field                   | Value                                                                                |
+-------------------------+--------------------------------------------------------------------------------------+
| admin_state_up          | UP                                                                                   |
| availability_zone_hints |                                                                                      |
| availability_zones      | nova                                                                                 |
| created_at              | 2018-01-03T21:36:11Z                                                                 |
| description             |                                                                                      |
| distributed             | False                                                                                |
| external_gateway_info   | {"network_id": "44b38c44-2a42-4b6d-b129-6c8f1b2a1375", \                             |
|                            "enable_snat": true, \                                                              |
|                            "external_fixed_ips": \                                                             |
|                           [{"subnet_id": "c3950603-9e04-4cc5-be8d-1efbfe59fc0a", "ip_address": "192.5.87.27"}]}| 
| flavor_id               | None                                                                                 |
| ha                      | False                                                                                |
| id                      | 8a9fef13-416b-4995-ba1c-cca588592862                                                 |
| name                    | router-physnet1-net-1                                                                | 
| project_id              | 63700f885ef14a8a8baf7766e275b189                                                     |
| revision_number         | 8                                                                                    |
| routes                  | destination='10.140.80.0/22', gateway='172.16.100.100’                               |
| status                  | ACTIVE                                                                               |
| tags                    |                                                                                      |
| updated_at              | 2018-01-03T21:36:45Z                                                                 |
+-------------------------+--------------------------------------------------------------------------------------+
 
 
 
### Remove Network
 
# Clean up router
openstack router unset --route destination=${DESTINATION},gateway=${NEXT_HOP} \
                       --external-gateway \
                       ${ROUTER_NAME}
openstack router remove subnet ${ROUTER_NAME} ${SUBNET_NAME}
openstack router delete ${ROUTER_NAME}
 
 
# Delete network
openstack network delete ${NET_NAME}