Kubernetes replica Testing

Here are some useful references
https://cloud.google.com/kubernetes-engine/docs/quickstart
https://cloud.google.com/kubernetes-engine/docs/tutorials/hello-app
https://kubernetes.io/blog/2016/07/autoscaling-in-kubernetes/

if you happen to share a project billing account with your friend, and you see error of no permission creating a bucket when you first push your image, do gcloud auth login

So testing effects of kubernetes replicas on running time
I am using google’s demo code

// [START all]
package main

import (
	"fmt"
	"log"
	"net/http"
	"os"
)

func main() {
	// register hello function to handle all requests
	mux := http.NewServeMux()
	mux.HandleFunc("/", hello)

	// use PORT environment variable, or default to 8080
	port := os.Getenv("PORT")
	if port == "" {
		port = "8080"
	}

	// start the web server on port and accept requests
	log.Printf("Server listening on port %s", port)
	log.Fatal(http.ListenAndServe(":"+port, mux))
}

// hello responds to the request with a plain-text "Hello, world" message.
func hello(w http.ResponseWriter, r *http.Request) {
	log.Printf("Serving request: %s", r.URL.Path)
	host, _ := os.Hostname()
	fmt.Fprintf(w, "Hello, world!\n")
	fmt.Fprintf(w, "Version: 1.0.0\n")
	fmt.Fprintf(w, "Hostname: %s\n", host)

  sum := 0
  for i := 0; i < 10000000000; i++ {
      sum += i
  }   
  fmt.Println("hello bobo")

}

// [END all]
# build the image and push it
docker build -t gcr.io/fuckctc/hello-app:v3
docker push gcr.io/fuckctc/hello-app:v3
# now create the kubenetes service
# the code runs 30s
kubectl create deployment hello-server3 --image=gcr.io/fuckctc/hello-app:v3
kubectl expose deployment hello-server3 --type LoadBalancer   --port 80 --target-port 8080

and we have our testing code

import time
from multiprocessing import Pool
import requests
def helper(url):
    return requests.get(url)

def main():
    t0 = time.time()
    pool = Pool(7)
    ### replace the ip with your service ip
    pool.map(helper, ['http://34.66.86.134/']*7)
    print(time.time() - t0)


if __name__ == "__main__":
    main()
# the code runs 17s
kubectl scale deployment hello-server2 --replicas=3

as I only have 2 nodes in the cluster, the running time reduce to ~17s
one caveat here is, the testing code has to be computing expensive, if you use sleep function, the server may simply get around with thread. which will be difficult to test

Docker create and push image

  1. create a dockerhub account and create repository
    here I call it “backend-server”
sudo docker run -it -d debian --name backend-server /bin/bash 
### setup the container ####
### sudo docker commit <containerID> <dockerhub repository>[:tag]
sudo docker commit backend-server tbjc1magic/backend-server

sudo docker login docker.io
### enter your dockerhub name and passwd ###

sudo docker push tbjc1magic/backend-server


setup https for wordpress with docker

1. install docker-ce

https://docs.docker.com/install/linux/docker-ce/debian/

2. install docker-compose

sudo curl -L "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

3. setup domain name, and link it to the node you are setting up your web server; install letsencrypt on the webserver

git clone https://github.com/certbot/certbot.git
cd certbot
sudo -H ./letsencrypt-auto certonly --standalone -d jianping-lai.com -d www.jianping-lai.com --email tbjc1magic@gmail.com
https://www.linode.com/docs/security/ssl/install-lets-encrypt-to-create-ssl-certificates/

4. install wordpress using docker with the docker-compose.yml below
https://docs.docker.com/compose/wordpress/; and do “sudo docker-compose up -d”

version: '3.3'

services:
   db:
     image: mysql:5.7
     volumes:
       - db_data:/var/lib/mysql
     restart: always
     environment:
       MYSQL_ROOT_PASSWORD: somewordpress
       MYSQL_DATABASE: wordpress
       MYSQL_USER: wordpress
       MYSQL_PASSWORD: wordpress

   wordpress:
     depends_on:
       - db
     image: wordpress:latest
     ports:
       - "80:80"
       - "443:443"
     restart: always
     environment:
       WORDPRESS_DB_HOST: db:3306
       WORDPRESS_DB_USER: wordpress
       WORDPRESS_DB_PASSWORD: wordpress
       WORDPRESS_DB_NAME: wordpress
volumes:
    db_data: {}

5. after start the wordpress docker container,

mkdir -p blog/etc sudo docker cp wordpress_wordpress_1:/etc/apache2 ~/blog/etc/

6. login to the wordpress docker container and do following

a2enmod ssl
service apache2 restart

7. shutdown the container and put the letsencrypt key and certificate combo to the ~/blog/etc/apache2 folder

cd ~/blog/etc/apache2/sites-enabled
sudo cp /etc/letsencrypt/live/jianping-lai.com/*.pem .
sudo ln -s ../sites-available/default-ssl.conf .
# modify the following two lines of default-ssl.conf
SSLCertificateFile      /etc/apache2/sites-enable/fullchain.pem
SSLCertificateKeyFile /etc/apache2/sites-enabled/privkey.pem
cd ~/blog/etc/apache2/mods-enabled
sudo ln -s ../mods-available/socache_shmcb.load .

8. modify the docker-compose.yml file add the following two lines

volumes:
    - /home/tbjc1magic/blog/etc/apache2:/etc/apache2

9. start docker containers again and turn on the ssl redirect plugin

sudo docker-compose up -d


Multiprocessing 1

I have been using python multiprocessing for quite a bit time, but it was all simple pool and I would like to understand deeper than just blindly using the multiprocessing package.

  1. given a list of word and start two process to print out the list in sequence
import multiprocessing
from multiprocessing import Pool, Queue, Process

def helper1(words, q1, q2):
   
    while 1:
        i = q1.get(timeout=1)
        if i >= len(words):
            q2.put(i+1)
            return
        print(words[i])
        q2.put(i+1)

def helper2(words, q1, q2):
   
    while 1:
        i = q2.get(timeout=1)
        if i >= len(words):
            q1.put(i+1)
            return

        print(words[i])
        q1.put(i+1)


def main():
   
    words = list(map(str, range(10)))
    q1 = Queue()
    q2 = Queue()
    q1.put(0)
    p1 = Process(target=helper1, args=(words, q1, q2), name='p1')
    p2 = Process(target=helper2, args=(words, q1, q2), name='p2')

    p1.start()
    p2.start()
    p1.join()
    p2.join()

The same functionality can also be achieved using pipe



import multiprocessing
from multiprocessing import Pool, Queue, Process, Pipe

def helper(words, pe, ps):
   
    while 1:
        i = pe.recv()
        if i >= len(words):
            ps.send(i+1)
            return
        print(words[i])
        ps.send(i+1)


def main():
   
    words = list(map(str, range(10)))
    p1e, p1s = Pipe(False)
    p2e, p2s = Pipe(False)
    
    p1s.send(0)
    p1 = Process(target=helper, args=(words, p1e, p2s), name='p1')
    p2 = Process(target=helper, args=(words, p2e, p1s), name='p2')

    p1.start()
    p2.start()
    p1.join()
    p2.join()


if __name__ == "__main__":
    main()

672. Bulb Switcher II

There is a room with n lights which are turned on initially and 4 buttons on the wall. After performing exactly m unknown operations towards buttons, you need to return how many different kinds of status of the n lights could be.

Suppose n lights are labeled as number [1, 2, 3 …, n], function of these 4 buttons are given below:

  1. Flip all the lights.
  2. Flip lights with even numbers.
  3. Flip lights with odd numbers.
  4. Flip lights with (3k + 1) numbers, k = 0, 1, 2, …

Example 1:

Input: n = 1, m = 1.
Output: 2
Explanation: Status can be: [on], [off]

Example 2:

Input: n = 2, m = 1.
Output: 3
Explanation: Status can be: [on, off], [off, on], [off, off]

Example 3:

Input: n = 3, m = 1.
Output: 4
Explanation: Status can be: [off, on, off], [on, off, on], [off, off, off], [off, on, on].

Note: n and m both fit in range [0, 1000].

class Solution(object):
    def flipLights(self, n, m):
        """
        :type n: int
        :type m: int
        :rtype: int
        """
        
        if m==0: return 1
        
        if n == 1: return 2
        
        
        def h2():
            status = [[0]*4 for _ in xrange(m+1)]
            
            status[0][3] = 1
            
            for i in range(1,m+1):
                for j,s in enumerate(status[i-1]):
                    if s: 
                        for idx in range(4):
                            if idx != j:
                                status[i][idx] = 1
                                
            return sum(status[m])
        
        def h3():
            status = [[0]*8 for _ in xrange(m+1)]
            
            status[0][7] = 1
            
            for i in range(1,m+1):
                for j,s in enumerate(status[i-1]):
                    if s:
                        status[i][j^7] = 1
                        status[i][j^5] = 1
                        status[i][j^2] = 1
                        status[i][j^1] = 1
                        
            return sum(status[m])
        
        
        if n==2: return h2()
        
        return h3()

 

 

670. Maximum Swap

Given a non-negative integer, you could swap two digits at most once to get the maximum valued number. Return the maximum valued number you could get.

Example 1:

Input: 2736
Output: 7236
Explanation: Swap the number 2 and the number 7.

Example 2:

Input: 9973
Output: 9973
Explanation: No swap.

Note:

  1. The given number is in the range [0, 108]
class Solution(object):
    def maximumSwap(self, num):
        """
        :type num: int
        :rtype: int
        """
        def helper(s):
            if not s: return ''
            d = max(s)
            if s[0] == d: return s[0]+helper(s[1:])
            
            for idx in xrange(len(s)-1,0,-1):
                if s[idx] == d:
                    return d+s[1:idx]+s[0]+s[idx+1:]
                
        
        return int(helper(str(num)))

 

669. Trim a Binary Search Tree

Given a binary search tree and the lowest and highest boundaries as L and R, trim the tree so that all its elements lies in [L, R] (R >= L). You might need to change the root of the tree, so the result should return the new root of the trimmed binary search tree.

Example 1:

Input: 
    1
   / \
  0   2

  L = 1
  R = 2

Output: 
    1
      \
       2

Example 2:

Input: 
    3
   / \
  0   4
   \
    2
   /
  1

  L = 1
  R = 3

Output: 
      3
     / 
   2   
  /
 1
class Solution(object):
    def trimBST(self, root, L, R):
        """
        :type root: TreeNode
        :type L: int
        :type R: int
        :rtype: TreeNode
        """
        def h1(r):
            if r is None: return None
            if r.val<L: 
                return h1(r.right)
            else:
                r.left = h1(r.left)
                return r
            
        def h2(r):
            if r is None: return None
            if r.val>R:
                return h2(r.left)
            else:
                r.right = h2(r.right)
                return r
            
        root = h1(root)
        root = h2(root)
        
        return root
            

671. Second Minimum Node In a Binary Tree

Given a non-empty special binary tree consisting of nodes with the non-negative value, where each node in this tree has exactly two or zero sub-node. If the node has two sub-nodes, then this node’s value is the smaller value among its two sub-nodes.

Given such a binary tree, you need to output the second minimum value in the set made of all the nodes’ value in the whole tree.

If no such second minimum value exists, output -1 instead.

Example 1:

Input: 
    2
   / \
  2   5
     / \
    5   7

Output: 5
Explanation: The smallest value is 2, the second smallest value is 5.

Example 2:

Input: 
    2
   / \
  2   2

Output: -1
Explanation: The smallest value is 2, but there isn't any second smallest value.
class Solution(object):
    def findSecondMinimumValue(self, root):
        """
        :type root: TreeNode
        :rtype: int
        """
        
        
        Min = root.val
        ans = 1e9
        stack = [root]
        
        while stack:
            nstack = []
            
            for n in stack:
                if n.val>Min:
                    ans = min(ans,n.val)
                    
                if n.left: 
                    nstack.append(n.left)
                    nstack.append(n.right)
                    

            
            stack = nstack
        
        
        if ans<1e9: return ans
        return -1

666. Path Sum IV

Given a list of ascending three-digits integers representing a binary with the depth smaller than 5. You need to return the sum of all paths from the root towards the leaves.
If the depth of a tree is smaller than 5, then this tree can be represented by a list of three-digits integers.
For each integer in this list:
  1. The hundreds digit represents the depth D of this node, 1 <= D <= 4.
  2. The tens digit represents the position P of this node in the level it belongs to, 1 <= P <= 8. The position is the same as that in a full binary tree.
  3. The units digit represents the value V of this node, 0 <= V <= 9.
Example 1:
Input: [113, 215, 221]
Output: 12
Explanation: 
The tree that the list represents is:
    3
   / \
  5   1

The path sum is (3 + 5) + (3 + 1) = 12.
Example 2:
Input: [113, 221]
Output: 4
Explanation: 
The tree that the list represents is: 
    3
     \
      1

The path sum is (3 + 1) = 4.

665. Non-decreasing Array

Example 1:

Input: [4,2,3]
Output: True
Explanation: You could modify the first 4 to 1 to get a non-decreasing array.

Example 2:

Input: [4,2,1]
Output: False
Explanation: You can't get a non-decreasing array by modify at most one element.

Note: The n belongs to [1, 10,000].

class Solution(object):
    def checkPossibility(self, nums):
        """
        :type nums: List[int]
        :rtype: bool
        """
        stack = []
        
        flag = False
        for n in nums:
            if not stack: 
                stack.append(n)
                continue
            
            if n < stack[-1]:
                if flag: return False
                
                if len(stack)<=1:
                    stack[-1] = min(n,stack[-1])
                else:
                    if n>stack[-2]:
                        stack[-1] = min(n,stack[-1])
                flag = True
            else:
                stack.append(n)
            
                
        return True