my net house

WAHEGURU….!

Category Archives: education

CRI-O(Container Runtime Interface)

CRIO for Kubernetes is what JRE for Java.

1. Kubernetes connects to kubelet to launch a POD.
POD: Pod is single bock for Kubernetes consisting of one or more container, sharing the same IPC, NET and PID namespaces and living in the same cgroup

2. Kubelet forwards the request to CRIO demon via CRI to launch new POD.

3. CRIO uses the container’s image library to pull the image from the container’s registry.  

4. downloaded image gets unpacked in container’s root file-system.(Just like we install some OS)  

5. after the rootfs is created CRI-O generates the OCI(Open Container interface) specification json-file explaining how to run Container.

6. Each container is maintained by a “conmon” process, it does monitoring,logging and handling PTY for the container.  

7. Networking for the pod is setup through use of CNI(Container network interface)

Now What is CNI and why do we need it?

 CNI concerns itself only with network connectivity of containers and removing allocated resources when the container is deleted.

Application containers on Linux are a rapidly evolving area, and within this area networking is not well addressed as it is highly environment-specific.

Sources:
https://github.com/containernetworking/cni
https://cri-o.io/

After reading the above, the following makes more sense.
https://cri-o.io/assets/images/architecture.png

Some more Aspects Kubernetes!!

  1. RepicationControler: it is “kind” of kubernetes which is responsible to take care about number of replications to use for service, container-image, Container-port, Service-name
  2. Service: It is responsible to make your Custer talk to outside world, It could be loadBalance, or Proxy service.

Now something a little more about Networking!!

  1. Cluster_IP-> it is the most basic type of service, An IP is assigned to service so Other services Could use this Service. Outside word can talk to your POD but wil have to go through some sort of PROXY!!
  2. TargetPort-> It allows you to separate the port from where you want to expose your service and where it is actually running inside container!!
  3. Nodeport-> it’s bit special and coulb be little scary as well but just hang-tight, nodeport makes service available to each node via statis port, so assume ou have three nodes, IP1, IP2, IP3 each are having same service running inside, You will assign some port number say 8081 and each node will be accessible on same port using Node’s IP and port. something like:

IP1:8081, IP2:8081, IP3:8081

4. ExternalIP: Another approach to making a service available outside of the cluster is via External IP addresses.

5. LoadBalancer: When running in the cloud, such as EC2 or Azure, it’s possible to configure and assign a Public IP address issued via the cloud provider. This will be issued via a Load Balancer such as ELB. This allows additional public IP addresses to be allocated to a Kubernetes cluster without interacting directly with the cloud provider.

For more one can follow following POST on linkedin!

https://www.linkedin.com/posts/dr-rabi-prasad-padhy-%E2%98%81%E2%98%81%E2%98%81-396804110_service-types-in-kubernetes-when-we-are-activity-6796285869864169472-gXF1

RUn Nginx in Kubernetes (Deployment and Scaling replicas) Part-2

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80

Take above yaml file as an example.

Now run Following command to create Deployment.

kubectl create -f nginx-depoy.yaml

Ways to check your pod has been Created or not.

khangura@metal-machine:~$ kubectl get pod

NAME READY STATUS RESTARTS AGE
hello-minikube-6ddfcc9757-k7qqv 1/1 Running 0 24h
nginx-deployment-5d59d67564-57sk7 1/1 Running 0 2m3s
nginx-deployment-5d59d67564-6d9t8 1/1 Running 0 2m3s
nginx-deployment-5d59d67564-bpsrj 1/1 Running 0 2m3s

We can always Update the deployment like increase/scale number of replicas.

Use following to update the Deployment.

kubectl edit deployment nginx-deployment # it will use our basic editor

Some Courses to Look into

Mathematics for Machine Learning.

Quantum Physics Full Course.

Linear Algebra Full Course.

Funtionaly Funtions in Pythonistic Python(s) by Pythonista! Part-2

6. having a wrapper outside and inside.

Importance of function wrapper is handling your data behaviour but also making sure , your wrapper is able to handle any kind of behaviour.

>>> def escape_unicode(f):                                                      
...     def wrap(*args, **kwargs):                                              
...         x = f(*args, **kwargs)                                              
...         return ascii(x)                                                     
...     return wrap                                                             
...                                                                             
>>> def northern_city():                                                        
...     return 'Tromsø'                                                         
...                                                                             
>>> print(northern_city())                                                      
Tromsø                                                                          
>>> @escape_unicode                                                             
... def northern_city():                                                        
...     return 'Tromsø'                                                         
...                                                                             
>>> print(northern_city())                                                      
'Troms\xf8'                                                                     

7. Using Class to create wapper/Decorator

Any class level attribute inside __call__() would be a Attribute for function in which Class is Wrapped around!

>>> class CallCount:                                                            
...     def __init__(self, f):                                                  
...         self.f = f                                                          
...         self.count = 0                                                      
...     def __call__(self, *args, **kwargs):                                    
...         self.count += 1                                                     
...         return self.f(*args, **kwargs)                                      
...                                                                             
>>> @CallCount                                                                  
... def hello(name):                                                            
...     print('Hello, {}!'.format(name))                                        
...                                                                             
>>> hello('Fred')                                                               
Hello, Fred!                                                                    
>>> hello('Wilma')                                                              
Hello, Wilma!                                                                   
>>> hello('Betty')                                                              
Hello, Betty!                                                                   
>>> hello('Barney')                                                             
Hello, Barney!                                                                  
>>> hello.count                                                                 
4

8. A wrapper inside class __call__() can be used to Turn Wrapper On and Off.

>>> class Trace:                                                                
...     def __init__(self):                                                     
...         self.enabled = True                                                 
...     def __call__(self, f):                                                  
...         def wrap(*args, **kwargs):                                          
...             if self.enabled:                                                
...                 print('Calling {}'.format(f))                               
...             return f(*args, **kwargs)                                       
...         return wrap                                                         
...                                                                             
>>> tracer = Trace()                                                            
>>> @tracer                                                                     
... @escape_unicode                                                             
... def norwegian_island_maker(name):                                           
...     return name + 'øy'                                                      
...                                                                             
>>> norwegian_island_maker('Llama')                                             
Calling <function escape_unicode.<locals>.wrap at 0x103b22ee0>                  
'Llama\\xf8y'                                                                   
>>> norwegian_island_maker('Python')                                            
Calling <function escape_unicode.<locals>.wrap at 0x103b22ee0>                  
'Python\\xf8y'                                                                  
>>> norwegian_island_maker('Troll')                                             
Calling <function escape_unicode.<locals>.wrap at 0x103b22ee0>                  
'Troll\\xf8y'                                                                   
>>> tracer.enabled = False                                                      
>>> norwegian_island_maker('Llama')                                             
'Llama\\xf8y'                                                                   
>>> norwegian_island_maker('Python')                                            
'Python\\xf8y'                                                                  
>>> norwegian_island_maker('Troll')                                             
'Troll\\xf8y'    

9. Map would be able to accept multiple arguents if Functions who is being mapped able to accept Multiple argument.

>>> colors = ['lavender', 'teal', 'burnt orange']                               
>>> animals = ['koala', 'platypus', 'salamander']                               
>>> def combine(size, color, animal):                                           
...     return '{} {} {}'.format(size, color, animal)                           
...                                                                             
>>> list(map(combine, sizes, colors, animals))                                  
['small lavender koala', 'medium teal platypus', 'large burnt orange salamander'
] 

10. List Comprehensions can be more lazy and complex with multipe For Loops and If statements.

>>> values = [x / (x - y) for x in range(100) if x > 50 for y in range(100) if x
 - y != 0]                                                                      
>>> values = [x / (x - y)                                                       
...           for x in range(100)                                               
...           if x > 50                                                         
...           for y in range(100)                                               
...           if x - y != 0]                                                    
>>> values = []                                                                 
>>> for x in range(100):                                                        
...     if x > 50:                                                              
...         for y in range(100):                                                
...             if x - y != 0:                                                  
...                 values.append(x / (x - y))                                  
...                                

11. list comprehensions could be Nested as well.

>>> vals = [[y * 3 for y in range(x)] for x in range(10)]                       
>>> outer = []                                                                  
>>> for x in range(10):                                                         
...     inner = []                                                              
...     for y in range(x):                                                      
...         inner.append(y * 3)                                                 
...     outer.append(inner)                                                     
...                                                                             
>>> vals                                                                        
[[], [0], [0, 3], [0, 3, 6], [0, 3, 6, 9], [0, 3, 6, 9, 12], [0, 3, 6, 9, 12, 15
], [0, 3, 6, 9, 12, 15, 18], [0, 3, 6, 9, 12, 15, 18, 21], [0, 3, 6, 9, 12, 15, 
18, 21, 24]]                

12. Some ideas for Code interospection!

  1. type() is of class type
>>> repr(int)                                                                   
"<class 'int'>"                                                                 
>>> type(i) is int                                                              
True                                                                            
>>> type(i)(78)                                                                 
78                                                                              
>>> type(type(i))                                                               
<class 'type'>                                                                  
>>> i.__class__                                                                 
<class 'int'>

2. isinstance and issubclass is type-checking subclass of class Type()

>>> issubclass(type, object)                                                    
True                                                                            
>>> type(object)                                                                
<class 'type'>                                                                  
>>> isinstance(i, int)                                                          
True                                                                         

3. To check if class object has specific attibute Eists.

 

>>> getattr(a, 'conjugate')                                                     
<built-in method conjugate of int object at 0x10ff2cfb0>                        
>>> callable(getattr(a, 'conjugate'))                                           
True                                                                            
>>> a.conjugate.__class__.__name__                                              
'builtin_function_or_method'                                                    
>>> getattr(a, 'index')                                                         
Traceback (most recent call last):                                              
  File "<stdin>", line 1, in <module>                                           
AttributeError: 'int' object has no attribute 'index'                           
>>> hasattr(a, 'bit_length')                                                    
True                                                                            
>>> hasattr(a, 'index')                                                         
False

4. Globas() and Locals() is Dict whch keep track of your globals and Locals.

>>> globals()                                                                   
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <cl
ass '_frozen_importlib.BuiltinImporter'>, '__spec__': None, '__annotations__': {
}, '__builtins__': <module 'builtins' (built-in)>}                              
>>> a = 42                                                                      
>>> globals()                                                                   
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <cl
ass '_frozen_importlib.BuiltinImporter'>, '__spec__': None, '__annotations__': {
}, '__builtins__': <module 'builtins' (built-in)>, 'a': 42}                     
>>> globals()['tau'] = 6.283185                                                 
>>> tau                                                                         
6.283185                                                                        
>>> tau / 2                                                                     
3.1415925                                                                       

>>> locals()                                                                    
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <cl
ass '_frozen_importlib.BuiltinImporter'>, '__spec__': None, '__annotations__': {
}, '__builtins__': <module 'builtins' (built-in)>}                              
>>> def report_scope(arg):                                                      
...     from pprint import pprint as pp                                         
...     x = 496                                                                 
...     pp(locals(), width=10)                                                  
...                                                                             
>>> report_scope(42)                                                            
{'arg': 42,                                                                     
 'pp': <function pprint at 0x10e70bee0>,                                        
 'x': 496}                                                                      
>>>                                                                             

5. import inspect.

Just like Golang’s reflect package.

>>> def num_vowels(text: str) -> int:                                           
...     return sum(1 if c.lower() in 'aeiou' else 0                             
...                for c in text)                                               
...                                                                             
>>> import inspect                                                              
>>> sig = inspect.signature(num_vowels)                                         
>>> sig.parameters['text']                                                      
<Parameter "text: str">                                                         
>>> sig.parameters['text'].annotation                                           
<class 'str'>                                                                   
>>> sig                                                                         
<Signature (text: str) -> int>                                                  
>>> sig.return_annotation                                                       
<class 'int'>                                                                   
>>> num_vowels.__annotations__                                                  
{'text': <class 'str'>, 'return': <class 'int'>}                                

 

Funtionaly Funtions in Pythonistic Python(s) by Pythonista! Part-1

  1. Call in Python:
    Yes I meant to Say __call__()

Every Function Object is invoked using __call__() method which is Dunder for Every Object in Python.

Example:

import socket
def resolve(host):
… return socket.gethostbyname(host)
…
resolve

resolve('google.com')
'172.217.24.238'
resolve('www.google.com')
'172.217.166.228'
resolve('gndec.ac.in')
'202.164.53.112'
resolve.call('gndec.ac.in')
'202.164.53.112'

2. Implement Local Cache for Class.

any function/object/variable start with _ {underScore} Could be used as Local/Private for that class, this could be used globally as well if defined using “global”

import socket

class Resolver:
    def __init__(self):
        self._cache = {}

    def __call__(self, host):
        if host not in self._cache:
            self._cache[host] = socket.gethostbyname(host)
        return self._cache[host]

>>>resolve = Resolver()
>>>resolve('sixty-north.com')
'93.93.131.30'
>>> resolve.__call__('sixty-north.com')
'93.93.131.30'
>>> resolve._cache
{'sixty-north.com': '93.93.131.30'}
>>> resolve('pluralsight.com')
'54.148.56.39'
>>> resolve._cache
{'sixty-north.com': '93.93.131.30', 'pluralsight.com': '54.148.56.39'}

3. Playing with “n” number of Keyword Args

def function(*args)

Imagine You want to Calculate Volume of shape and it could be Square, Cube, Tesseract or anything available even in Marvel universe.

>>> def hypervolume(*args):                                                     
...     print(args)                                                             
...     print(type(args))                                                       
...                                                                             
>>> hypervolume(3, 4)                                                           
(3, 4)                                                                          
<class 'tuple'>                                                                 
>>> hypervolume(3, 4, 5)                                                        
(3, 4, 5)                                                                       
<class 'tuple'>                                                                 
>>> def hypervolume(*lengths):                                                  
...     i = iter(lengths)                                                       
...     v = next(i)                                                             
...     for length in i:                                                        
...         v *= length                                                         
...     return v                                                                
...                                                                             
>>> hypervolume(2, 4)                                                           
8                                                                               
>>> hypervolume(2, 4, 6)                                                        
48                                                                              
>>> hypervolume(2, 4, 6, 8)                                                     
384

 4. Function Enclosing.

Every Function object is Returnable Just like another function. This is also called closures.

>>> def raise_to(exp):
...    def raise_to_exp(x):
...        return pow(x,exp)
...    return raise_to_exp
... 
>>> square = raise_to(2)
>>> square
<function raise_to_exp at 0x7f9d0f6da950>
>>> square(9)
81
>>> qube = raise_to(3)
>>> qube(3)
27
>>> qube(27)
19683
>>> 

Now first time you have set Default value for Expression/Object/Function. Any further cal will apply that value on your data. Very useful while writing default behaviour for API calls, or DB calls.

Easy way to have Module Support in Golang

In Your main.go file add all the dependencies, like “yourProject/lib”, “yourProject/test”,”yourProject/hello”

The go to Root DIR of project and run Following:

go mod init yourProject ..> it will generate go.mod file

go mod download ..> it will generate go.sum file

go mod download

go mod verify
That’s all , enjoy life and amazing Module level management in Golang.

 

Install Vanilla Kubernetes Cluster on Ubuntu.

Here We are Installing Vanilla Flavour of Kubernetes Cluster(With High Availablility), Which is Small but Production-Ready.

 

Prerequisites:

 

  • You need to have at least Two VMs(Master Node and Slave Node)
  • Both VMs should connected on the Same Network.
  • Master Node must have 2 CPUs and at least 2 GB of RAM.
  • Swap must be turned OFF on Both of the VM(s).
  • A basic understanding of Linux, Networking and docker unless you are a magician. 😉

 

Installation Steps:

Run Following Command on Master Node.(Without Hash 😛 ) 
#  apt install docker.io

# systemctl enable docker

# apt install curl

# curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add

# apt-add-repository “deb http://apt.kubernetes.io/ kubernetes-xenial main”

# apt install kubeadm

After Running all the Commands above you would be able to Install All Kubernetes Packages on the Master Node.

Now you need to Deploy Cluster. Run Following Commands. 

 

# swapoff -a

#  hostnamectl set-hostname master-node

# kubeadm init –pod-network-cidr=10.244.0.0/16 (Look at it’s output Carefully!!)

# mkdir -p $HOME/.kube

# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

# chown $(id -u):$(id -g) $HOME/.kube/config

 

 

Deploy Pod Network: (There are different Types of Networks you can use. Here we are using flannel)

 

$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

 

Add the slave node to the network in order to form a cluster. 

Run the Command which you will get in output after running this:
kubeadm init –pod-network-cidr=10.244.0.0/16” on you Master Node. make sure you have to run Command which will lokk like Following on Your Slave-node. So you can use Kubernetes Cluster. 

** Don’t Run Following Command as it is.

 

kubeadm join 172.31.9.252:6443 –token bixhtx.4bqyhe45g8fvhqeq \

    –discovery-token-ca-cert-hash sha256:eb5f21bfda175f8d60ca50780941df83169893182c8fd2073de32daccc7b2e6d

 

Other Comman Kubernetes commands to Play with your Cluster:

# kubectl get nodes

# kubectl get pods –all-namespaces

 

Enjoy your Cluster!! 

Courses to complete this Quarter

Understand Stack and Heap from the Ideology of C and C/C++

Writing Flow:
1. Introduction to working of Stack and Heap inside Memory .

2. How Stack Works.

3. How Heap Works.

4. Conclusion based on performance and Ease of use. ::

There are various parts which are responsible to make one complete set of memory which is responsible for Execution and life cycle of application  or process. We can also say that this process is unified either you are using ThreadPool or Process Pool. Whole memory is divided into major three parts, Stack, Heap, Code(Text), Global variables and Constant’s Section, As from the name of last two it can be concluded easily that these parts are responsible for storage of Global Variable and keeping the source code, Other two important parts left are Stack and Heap. Let’s talk about Stack in next Section.

 

Stack is container of one and many functions with the static memory allocated to it, All the memory to stack is allocated before the compile time and it remains same throughout the whole life cycle of application, if there comes more requirement of memory than the allocated one it becomes famous “Stack-over-flow. Main() function call sits on the bottom of the stack. It gets executed at the end of the Stack, All the other Function based on calls sit on each other  in the stack. All the local Variables are also stored in the Stack as well.

 

Heap is something different in terms of ideology from the stack, Where stack is all about occurring of function call  in the series with fixed memory size but in the case of Heap it is all Dynamic. When we need requirement of memory in the runtime without not fixed size heap is there for us. There are multiple ways we can use memory allocations in heap in C and C++. Some known methods are Malloc(), Calloc(), realloc() and free(). C++ is superset of C so all the mentioned functions can also be used in C++ as well, with each call of these functions we get Pointer to Memory block, with that Block one can store data. Make sure one this that in C and C++ memory allocation is not Dynamic so one has to Allocate and De-Allocate memory manually.

Apart from C and C++ there are many other languages those make substantial use of Stacks and Heaps, For example in Python most of the stuff is Stored on heap so  you can allocated data to it even on run time but when you use Generators() or Lazy-Iterators() those are framed on stack and with each continuous  call each functions free the space in the Stack, In the case of Go-lang  something very weird happens, Quoting from StackOverFlow-

It’s worth noting that the words “stack” and “heap” do not appear anywhere in the language spec. Your question is worded with “…is declared on the stack,” and “…declared on the heap,” but note that Go declaration syntax says nothing about stack or heap.

That technically makes the answer to all of your questions implementation dependent. In actuality of course, there is a stack (per goroutine!) and a heap and some things go on the stack and some on the heap. In some cases the compiler follows rigid rules (like “new always allocates on the heap”) and in others the compiler does “escape analysis” to decide if an object can live on the stack or if it must be allocated on the heap.

Source: https://stackoverflow.com/questions/10866195/stack-vs-heap-allocation-of-structs-in-go-and-how-they-relate-to-garbage-collec

 

Source for Stack and Heap of C/C++: https://www.youtube.com/watch?v=_8-ht2AKyH4

 

Note:* Overall  Performance and Consistency of the Application is based on smart use of both Stack and Heap. 

 

 

%d bloggers like this: