Calico IPAM in Rancher v2

The rancher is a great setup with easy-to-config and magnificent UI and integration with cloud IaaS providers like AWS. I have been a fan of the project since the early state of Rancher in version 1 with cattle style deployment before kubernetes become standard for the community of container orchestration.
Calico is another CNI with many features and flexibility that make it a great choice for cloud environments in pure L3 network level.

This article depends on your networking vision and requirements in the kubernetes cluster, so reading the following articles is necessary for a deep understanding of networking in kubernetes.

Cluster Networking

Configuring the Calico CNI plugins

Calico Architecture

Flannel vs Calico : A battle of L2 vs L3 based networking

VXLAN explained

Understanding kubernetes networking: ingress

Understanding kubernetes networking: services

Why calico-ipam?

The rancher by default uses the host-local calico plugin which itself uses kubernetes API CIDR address for assigning an IP address to containers. This is a painless simple network for small size k8s clusters but in a complicated scenario, we need different options, such as IP ranges or floating IP address for deployments that we cannot load balance or HA them with nginx ingress.

How to implement calico-ipam in rancher?

In order to use calico-ipam in rancher, we need to have another etcd cluster and to edit some of the variables in calico deployment for kubernetes.

Etcd is installed by RKE(Rancher Kubernetes Engine) in your host, so you can use this secure setup in calico by mounting key and certs in the calico controller and calico-node and address them in configmap or simply put them into configmap or secret , due the educational purpose in this tutorial I using another not-secure etcd pod instead of rke secure etcd.

I put both of calico and etcd deployment in this gist.

important note :

default CIDR of the rancher is 10.43.0.0/16 but if you use another setup tool you can see CIDR with

kubectl cluster-info dump | grep -i cidr

final test and results

I am using the latest current version of calico that is v3.5 , this version support namespace annotation and nodeselector for ippool

for creating rolls we need calicoctl and connecting calicoctl to etcdv3 backend of calico I prefer alias in my shell

alias calicoctl="ETCD_ENDPOINTS=http://YourEtcdHostIp:2382 ~/./calicoctl"

now the time is create ippool and annotate them to the namespaces

apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
  name: external-pool
spec:
  cidr: 172.16.0.0/26
  blockSize: 29
  ipipMode: Always
  natOutgoing: true
---
apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
  name: internal-pool
spec:
  cidr: 192.169.0.0/24
  blockSize: 29
  ipipMode: Always
  natOutgoing: true

and after create this file simply apply to cluster by

calicoctl appy -f test.yml

now you can see your ippool in etcd with

calicoctl get ipPool

and annotate ippools to namespaces

kubectl annotate namespace external-ns "cni.projectcalico.org/ipv4pools"="["external-pool"]"
kubectl annotate namespace internal-ns "cni.projectcalico.org/ipv4pools"="["internal-pool"]"

if everything is ok your new deployment in two namespaces most get a different ip address.

further reading

calico have many options for securing or connecting namespaces and containers with Network Policy and Global Network Policy and scale cluster with BGP route which is great compared to flannel.