OpenStack on Kubernetes (GitHub)

Cloud 2018. 8. 16. 23:14

GitHub

https://github.com/call518/OpenStack-on-Kubernetes

Greetings

Hi! all, Welcome to my project. I hope this will be useful thing. :)

p.s. And I hope that some guy help me. If you are interested in this, please contact me. thank you.

Intro

OaaS is "OpenStack as a Service".
MAINTAINER: Jung-In.Jung (call518@gmail.com)
2018-06-20 ~ Now

Goals

Auto-Provisioning OpenStack, Based on Kubernetes(k8s).
Scalaing for LB/HA.
Supoort Provisioning Almost OpenStack Releases. (Newton/Ocata/Pike/Queens/...)
Simple and Dynamic Configuration.
Auto Failover/Failback.

Components Diagram

Networking Diagram

Progress

Ocata
- memcached (Completed)
- rabbitmq (Completed)
- mongodb (Completed)
- etcd (Completed)
- galera (Completed)
- haproxy (Completed)
- keystone (Completed)
- glance (Completed)
- nova (Completed)
- neutron (Completed)
- cinder (Completed)
- heat (Completed)
- ceilometer-central (Completed)
- aodh (Completed)
- horizon (Completed)
Integration of OpenStack Releases.
- (TODO: Planning...)

Tutorial

System Env. & Arch.

Requirements.

General Kubernetes Cluster.
- kubernetes Installation guide
All k8s Worker nodes have to sync Time (e.g. chrony, ntp)
k8s worker nodes for neutron-server/nova-compute need to load openvswitch/ebtables/ip_vs kernel module.
- Run contents of host_kernel_modules_for_oaas.sh on compute/network role(label) woker nodes.
Quorum PODs (Replica is have to 2n+1)
- galera-etcd
- galera
- rabbitmq

Env.

(Note) We have tested this tutorial with 6 VMs on VirtualBox Env. If u want, any env is possible. (eg. physical machines)

(Note) We use NFS for cinder backend storage for simple tutorial, but soon we will change to ceph back-end storage.

Spec. of Physical Host

Processor: Intel Core i5-6500 (3.2GHz)
Memory: 32GB
Storage: SSD (2TB)
NIC: Intel(R) Ethernet Connection (5) I219-LM

Spec. of each VM

(Note) Maybe, you need big RAM. if not, reduce number of replicas.

CentOS-7 x86_64
2 EA vCPUs
vRAM: 2GB ~ 6GB
100 GB vDisk
1 EA NIC

Spec. of k8s

This is versions of packages that i have tested.

docker-ce-18.06.0.ce-3.el7.x86_64
kubernetes-cni-0.6.0-0.x86_64
kubectl-1.11.1-0.x86_64
kubelet-1.11.1-0.x86_64
kubeadm-1.11.1-0.x86_64

VM OS Env.

/etc/hosts

# cat /etc/hosts

192.168.0.150 k8s-master
192.168.0.151 k8s-node01
192.168.0.152 k8s-node02
192.168.0.153 k8s-node03
192.168.0.154 k8s-node04
192.168.0.155 k8s-node05

Deploy Tutorial

Node Labels (Role)

(!) Network(neutron-sever) worker nodes must be separated/dedicated. (network=true)

[k8s-master]# kubectl get node --show-labels

NAME         STATUS    ROLES     AGE       VERSION   LABELS
k8s-master   Ready     master    8d        v1.11.1   node-role.kubernetes.io/master=
k8s-node01   Ready     <none>    8d        v1.11.1   controller=true,compute=true,nfs-server=true
k8s-node02   Ready     <none>    8d        v1.11.1   controller=true,compute=true
k8s-node03   Ready     <none>    8d        v1.11.1   controller=true,compute=true
k8s-node04   Ready     <none>    8d        v1.11.1   network=true
k8s-node05   Ready     <none>    8d        v1.11.1   network=true

default configs (eg. password)

check main env file, src-ocata/configMap-env-common.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: env-common
data:
  K8S_OPENSTACK_RELEASE: "ocata"
  MYSQL_ROOT_PASSWORD: "passw0rd"
  DISCOVERY_SERVICE: "etcd-client:2379"
  XTRABACKUP_PASSWORD: "passw0rd"
  CLUSTER_NAME: "mariadb_galera_ss"
  #MYSQL_DATABASE: "mydatabase"
  #MYSQL_USER: "myuser"
  MYSQL_PASSWORD: "passw0rd"
  K8S_MONGO_USER: "admin"
  K8S_MONGO_PASS: "passw0rd"
  K8S_KEYSTONE_DB_PASS: "passw0rd"
  K8S_GLANCE_DB_PASS: "passw0rd"
  K8S_CINDER_DB_PASS: "passw0rd"
  K8S_NEUTRON_DB_PASS: "passw0rd"
  K8S_NOVA_DB_PASS: "passw0rd"
  K8S_GNOCCHI_DB_PASS: "passw0rd"
  K8S_AODH_DB_PASS: "passw0rd"
  K8S_HEAT_DB_PASS: "passw0rd"
  K8S_KEYSTONE_USER_ADMIN_PASS: "passw0rd"
  K8S_KEYSTONE_USER_DEMO_PASS: "demo"
  K8S_KEYSTONE_USER_GLANCE_PASS: "passw0rd"
  K8S_KEYSTONE_USER_CINDER_PASS: "passw0rd"
  K8S_KEYSTONE_USER_NEUTRON_PASS: "passw0rd"
  K8S_KEYSTONE_USER_NOVA_PASS: "passw0rd"
  K8S_KEYSTONE_USER_PLACEMENT_PASS: "passw0rd"
  K8S_KEYSTONE_USER_GNOCCHI_PASS: "passw0rd"
  K8S_KEYSTONE_USER_CEILOMETER_PASS: "passw0rd"
  K8S_KEYSTONE_USER_AODH_PASS: "passw0rd"
  K8S_KEYSTONE_USER_HEAT_PASS: "passw0rd"
  K8S_HAPROXY_STATS_USERNAME: "admin"
  K8S_HAPROXY_STATS_PASSWORD: "passw0rd"
  K8S_RABBITMQ_ADMIN_USER: "admin"
  K8S_RABBITMQ_ADMIN_PASS: "passw0rd"
  K8S_RABBITMQ_OPENSTACK_USER: "openstack"
  K8S_RABBITMQ_OPENSTACK_PASS: "passw0rd"
  K8S_METADATA_PROXY_SHARED_SECRET: "QU6muuXhU4oAeLzDas6obGsDtoFNZTHq"
  K8S_NFS_SERVER_IP_ETC_KEY: "k8s-oaas-nfs-server-ip-address"
  K8S_EXT_SUBNET_CIDR: "192.168.100.0/24"
  K8S_EXT_SUBNET_POOL_START: "192.168.100.101"
  K8S_EXT_SUBNET_POOL_END: "192.168.100.200"
  K8S_EXT_SUBNET_GW: "192.168.100.1"
  K8S_DEMO_SUBNET_CIDR: "172.16.0.0/24"
  K8S_DEMO_SUBNET_GW: "172.16.0.1"
  K8S_DEMO_SUBNET_DNS: "8.8.8.8"

Required Docker Images.

call518/oaas-init-container
call518/oaas-nfs-server
call518/oaas-etcd
call518/oaas-galera
call518/oaas-memcached
call518/oaas-rabbitmq
call518/oaas-mongodb
call518/oaas-haproxy
call518/oaas-zookeeper
call518/oaas-ocata

Initiate Deploying OpenStack

[k8s-master]# git clone [here]

[k8s-master]# cd OpenStack-on-Kubernetes/src-ocata

[k8s-master]# ./start-oaas.sh

Result Deploying

[k8s-master]# kubectl get all -o wide

NAME                           READY     STATUS    RESTARTS   AGE       IP             NODE
pod/cinder-0                   1/1       Running   0          38m       10.244.3.130   k8s-node03
pod/cinder-1                   1/1       Running   0          6m        10.244.1.143   k8s-node01
pod/etcd0                      1/1       Running   0          38m       10.244.2.112   k8s-node02
pod/etcd1                      1/1       Running   0          38m       10.244.3.127   k8s-node03
pod/etcd2                      1/1       Running   0          38m       10.244.1.137   k8s-node01
pod/galera-0                   1/1       Running   1          38m       10.244.2.117   k8s-node02
pod/galera-1                   1/1       Running   0          17m       10.244.3.133   k8s-node03
pod/galera-2                   1/1       Running   1          33m       10.244.1.142   k8s-node01
pod/glance-0                   1/1       Running   0          38m       10.244.1.140   k8s-node01
pod/glance-1                   1/1       Running   0          10m       10.244.3.135   k8s-node03
pod/haproxy-7b567f67d8-mxm7v   1/1       Running   3          38m       10.244.3.128   k8s-node03
pod/horizon-6965547f56-kgc44   1/1       Running   0          38m       10.244.2.116   k8s-node02
pod/keystone-0                 1/1       Running   0          38m       10.244.2.114   k8s-node02
pod/keystone-1                 1/1       Running   0          13m       10.244.3.134   k8s-node03
pod/memcached-0                1/1       Running   0          38m       10.244.2.113   k8s-node02
pod/memcached-1                1/1       Running   0          38m       10.244.3.129   k8s-node03
pod/memcached-2                1/1       Running   0          38m       10.244.1.138   k8s-node01
pod/neutron-server-0           1/1       Running   0          38m       10.244.1.139   k8s-node01
pod/neutron-server-1           1/1       Running   0          5m        10.244.3.136   k8s-node03
pod/nfs-server                 1/1       Running   0          38m       10.244.1.136   k8s-node01
pod/nova-compute-0             1/1       Running   0          38m       10.244.4.24    k8s-node04
pod/nova-compute-1             1/1       Running   0          38m       10.244.5.24    k8s-node05
pod/nova-server-0              1/1       Running   0          38m       10.244.2.115   k8s-node02
pod/nova-server-1              1/1       Running   0          4m        10.244.1.144   k8s-node01
pod/rabbitmq-0                 1/1       Running   0          38m       10.244.3.131   k8s-node03
pod/rabbitmq-1                 1/1       Running   0          35m       10.244.1.141   k8s-node01
pod/rabbitmq-2                 1/1       Running   0          35m       10.244.2.118   k8s-node02

NAME                          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                                                       AGE       SELECTOR
service/cinder                ClusterIP   None             <none>        8776/TCP                                                      38m       app=cinder
service/etcd-client           ClusterIP   10.109.90.101    <none>        2379/TCP                                                      38m       app=etcd
service/etcd0                 ClusterIP   10.110.11.231    <none>        2379/TCP,2380/TCP                                             38m       etcd_node=etcd0
service/etcd1                 ClusterIP   10.98.35.253     <none>        2379/TCP,2380/TCP                                             38m       etcd_node=etcd1
service/etcd2                 ClusterIP   10.110.157.230   <none>        2379/TCP,2380/TCP                                             38m       etcd_node=etcd2
service/galera                ClusterIP   None             <none>        3306/TCP                                                      38m       app=galera
service/glance                ClusterIP   None             <none>        9292/TCP,9191/TCP                                             38m       app=glance
service/haproxy-galera        ClusterIP   None             <none>        3306/TCP                                                      38m       app=haproxy
service/haproxy-stats         NodePort    10.103.245.255   <none>        9000:30090/TCP                                                38m       app=haproxy
service/horizon               NodePort    10.103.67.208    <none>        80:30080/TCP                                                  38m       app=horizon
service/keystone              ClusterIP   None             <none>        5000/TCP,35357/TCP                                            38m       app=keystone
service/kubernetes            ClusterIP   10.96.0.1        <none>        443/TCP                                                       2d        <none>
service/memcached             ClusterIP   None             <none>        11211/TCP                                                     38m       app=memcached
service/neutron-server        ClusterIP   None             <none>        9696/TCP                                                      38m       app=neutron-server
service/nova-compute          ClusterIP   None             <none>        8774/TCP,8775/TCP,6080/TCP                                    38m       app=nova-compute
service/nova-server           NodePort    10.111.84.90     <none>        8774:30177/TCP,8778:30246/TCP,8775:31964/TCP,6080:30068/TCP   38m       app=nova-server
service/rabbitmq              ClusterIP   None             <none>        5672/TCP,4369/TCP,25672/TCP                                   38m       app=rabbitmq
service/rabbitmq-management   ClusterIP   None             <none>        15672/TCP                                                     38m       app=rabbitmq

NAME                      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE       CONTAINERS   IMAGES                 SELECTOR
deployment.apps/haproxy   1         1         1            1           38m       haproxy      call518/oaas-haproxy   app=haproxy
deployment.apps/horizon   1         1         1            1           38m       horizon      call518/oaas-ocata     app=horizon

NAME                                 DESIRED   CURRENT   READY     AGE       CONTAINERS   IMAGES                 SELECTOR
replicaset.apps/haproxy-7b567f67d8   1         1         1         38m       haproxy      call518/oaas-haproxy   app=haproxy,pod-template-hash=3612392384
replicaset.apps/horizon-6965547f56   1         1         1         38m       horizon      call518/oaas-ocata     app=horizon,pod-template-hash=2521103912

NAME                              DESIRED   CURRENT   AGE       CONTAINERS       IMAGES
statefulset.apps/cinder           2         2         38m       cinder           call518/oaas-ocata
statefulset.apps/galera           3         3         38m       galera           call518/oaas-galera
statefulset.apps/glance           2         2         38m       glance           call518/oaas-ocata
statefulset.apps/keystone         2         2         38m       keystone         call518/oaas-ocata
statefulset.apps/memcached        3         3         38m       memcached        call518/oaas-memcached
statefulset.apps/neutron-server   2         2         38m       neutron-server   call518/oaas-ocata
statefulset.apps/nova-compute     2         2         38m       nova-compute     call518/oaas-ocata
statefulset.apps/nova-server      2         2         38m       nova-server      call518/oaas-ocata
statefulset.apps/rabbitmq         3         3         38m       rabbitmq         call518/oaas-rabbitmq

Open Horizon Dashboard

In browser, http://[One_of_worker_nodes]:30080

Scaling

(Note) Maybe, needed more worker nodes...

<Examples>

[k8s-master]# kubectl scale --replicas=4 statefulset.apps/cinder
[k8s-master]# kubectl scale --replicas=5 statefulset.apps/galera (must 2n+1)

ScreenShots

Login

Overview

Instances

Network Topology

Instance Web Console

TODO

Re-Configuration Flat-IP Ranbe for EXT-NET.
(Done) ~~Simplify initContainer Check-Processing.~~
Integration of All of Provision Source. /w Template and ETc...

Appendix

Repositories

Active Repository (GitHub)

https://github.com/call518/OpenStack-on-Kubernetes

Mirrored Repository (GitLab)

https://gitlab.com/call518/OpenStack-on-Kubernetes

References

저작자표시

'Cloud' 카테고리의 다른 글

Concept of IaaS, PaaS, SaaS (0)	2017.09.20
Orchestration Docker on Mesos/Marathon (0)	2015.03.14
SDN Test Suite /w Vagrant (0)	2014.12.23
OpenStack - How To Install DevStack with Vagrant/VirtualBox (0)	2014.03.12
[Open] EYWA - Detailed Information (0)	2014.03.03

Posted by 사랑줍는거지

Concept of IaaS, PaaS, SaaS

Cloud 2017. 9. 20. 10:27

Goal은 똑같다... "밥먹자!!"

SaaS "Just enjoy!"

PaaS "Cooking is my pleasure!"

IaaS "Ooooops!, OMG!!!, I'm hungry!!"

(철지난 개념 정리를 또 끄적이고 떠들게 될줄이야..... 그것도 자칭 클라우드 전문가에게....... 싫타 정말.... IaaS, PaaS, SaaS... 구분짓는다는게 참 우스운 일이긴 하지만, 어쩌겄어... 대화를 시작이라도 해볼려면... 쿨럭...)

저작자표시 비영리 변경금지

'Cloud' 카테고리의 다른 글

OpenStack on Kubernetes (GitHub) (0)	2018.08.16
Orchestration Docker on Mesos/Marathon (0)	2015.03.14
SDN Test Suite /w Vagrant (0)	2014.12.23
OpenStack - How To Install DevStack with Vagrant/VirtualBox (0)	2014.03.12
[Open] EYWA - Detailed Information (0)	2014.03.03

Posted by 사랑줍는거지

Orchestration Docker on Mesos/Marathon

Cloud 2015. 3. 14. 00:15

Orchestration Docker on Mesos/Marathon from JungIn Jung

저작자표시 비영리 변경금지

'Cloud' 카테고리의 다른 글

OpenStack on Kubernetes (GitHub) (0)	2018.08.16
Concept of IaaS, PaaS, SaaS (0)	2017.09.20
SDN Test Suite /w Vagrant (0)	2014.12.23
OpenStack - How To Install DevStack with Vagrant/VirtualBox (0)	2014.03.12
[Open] EYWA - Detailed Information (0)	2014.03.03

Posted by 사랑줍는거지

이전 1 2 3 4 ··· 22 다음


	Cloud Story... by 사랑줍는거지

« 2024/11 »
일	월	화	수	목	금	토
					1	2
3	4	5	6	7	8	9
10	11	12	13	14	15	16
17	18	19	20	21	22	23
24	25	26	27	28	29	30

사랑줍는 거지

OpenStack on Kubernetes (GitHub)

Greetings

Intro

Goals

Components Diagram

Networking Diagram

Progress

Tutorial

System Env. & Arch.

Requirements.

Env.

Spec. of Physical Host

Spec. of each VM

Spec. of k8s

VM OS Env.

/etc/hosts

Deploy Tutorial

Node Labels (Role)

default configs (eg. password)

Required Docker Images.

Initiate Deploying OpenStack

Result Deploying

Open Horizon Dashboard

Scaling

ScreenShots

Login

Overview

Instances

Network Topology

Instance Web Console

TODO

Appendix

Repositories

Active Repository (GitHub)

Mirrored Repository (GitLab)

References

'Cloud' 카테고리의 다른 글

Concept of IaaS, PaaS, SaaS

'Cloud' 카테고리의 다른 글

Orchestration Docker on Mesos/Marathon

'Cloud' 카테고리의 다른 글

카테고리

태그목록

최근에 올라온 글

최근에 받은 트랙백

달력

링크

티스토리툴바