use ssh for drone to deploy app on instance.

We can instruct drone to deploy the application on an instance/VM with providing the access.
->
==
Add hostname, user and password as drone secrets.

drone secret add --image appleboy/drone-ssh --repository <git repo name> --name ssh_user --value <username>
drone secret add --image appleboy/drone-ssh --repository <git repo name> --name ssh_host --value 'IP/HostName'
drone secret add --image appleboy/drone-ssh --repository <git repo name> --name ssh_key --value @/<Key path>

==

.drone.yaml

---
workspace: /* help to clone the repo to drone container and same repo can scp to an instance/VM */
  base: /go
  path: </tmp>
pipeline:
  ssh-connection:
    image: appleboy/drone-ssh
    secrets: [ ssh_host, ssh_user, ssh_key ]
    commands:
      - echo $${SSH_HOST}
      - echo $${SSH_USER}
      - . ./ssh_setup.sh
      - . ./test_script.sh $${SSH_USER} $${SSH_HOST}
    when:
      event: [ push ]




ssh_setup.sh

#!/bin/bash
eval $(ssh-agent -s)
printf '%b%s' "$SSH_KEY" | ssh-add -
#echo $${SSH_KEY} > /tmp/SSH_KEY && chmod 600 /tmp/SSH_KEY

test_script.sh

#!/bin/bash
SSH_USER=$1
SSH_HOST=$2
#scp the repo from drone workspace
scp -r -q -o StrictHostKeyChecking=no </tmp>  ${SSH_USER}@${SSH_HOST}:/tmp/
#scp the script to /tmp
scp -q -o StrictHostKeyChecking=no <any thing> ${SSH_USER}@${SSH_HOST}:/tmp/
#Excute the script.sh from the server
ssh -q -tt -o StrictHostKeyChecking=no ${SSH_USER}@${SSH_HOST} "sudo /bin/sh <anything>"

understanding test kitchen & its cheat sheet.

reference# https://kitchen.ci/docs/getting-started/creating-cookbook
==
-> Reads the .kitchen.yaml file inside the cookbook. It looks like below.

---
driver:
  name: vagrant -> tells the driver may vagrant/ec2..

provisioner:
  name: chef_zero 

verifier:
  name: inspec

platforms:
  - name: ubuntu-16.04

suites:
  - name: default
    run_list:
      - recipe[git_cookbook::default]
    verifier:
      inspec_tests:
        - test/smoke/default
    attributes:

Commands...

 kitchen console                                 # Kitchen Console!
 kitchen converge [INSTANCE|REGEXP|all]          # Change instance state to converge. Use a provisioner to configure one or more instances
 kitchen create [INSTANCE|REGEXP|all]            # Change instance state to create. Start one or more instances
 kitchen destroy [INSTANCE|REGEXP|all]           # Change instance state to destroy. Delete all information for one or more instances
 kitchen diagnose [INSTANCE|REGEXP|all]          # Show computed diagnostic configuration
 kitchen driver                                  # Driver subcommands
 kitchen driver create [NAME]                    # Create a new Kitchen Driver gem project
 kitchen driver discover                         # Discover Test Kitchen drivers published on RubyGems
 kitchen driver help [COMMAND]                   # Describe subcommands or one specific subcommand
 kitchen exec INSTANCE|REGEXP -c REMOTE_COMMAND  # Execute command on one or more instance
 kitchen help [COMMAND]                          # Describe available commands or one specific command
 kitchen init                                    # Adds some configuration to your cookbook so Kitchen can rock
 kitchen list [INSTANCE|REGEXP|all]              # Lists one or more instances
 kitchen login INSTANCE|REGEXP                   # Log in to one instance
 kitchen package INSTANCE|REGEXP                 # package an instance
 kitchen setup [INSTANCE|REGEXP|all]             # Change instance state to setup. Prepare to run automated tests. Install busser and related gems on one or more instances
 kitchen test [INSTANCE|REGEXP|all]              # Test (destroy, create, converge, setup, verify and destroy) one or more instances
 kitchen verify [INSTANCE|REGEXP|all]            # Change instance state to verify. Run automated tests on one or more instances
 kitchen version         

Docker multi container with Python and mysql

This set up describes about a multi docker container setup with python and mysql.

We will create two container, one for Python App and another for Mysql and will the communication between each.

==> Create a Dockerfile for mysql-server.

#Pull Docker image.

FROM mysql:5.6

ADD nsebackup.sql /nsebackup.sql

==> Build the image

#docker build -t mysql-local-image .

==> Create the conatiner with local mouneted volume. Create the local directory on host.

#mkdir /datadir

# docker run -d --name mysql-local-server -p 3306:3306 -v  /datadir:/var/lib/mysql -e  MYSQL_ROOT_PASSWORD=xxx mysql-local-image

==> you should  able to see the container "mysql-local-server" running on the hsot using docker ps commnd.

#### Create the python app container####

==> Create a Dockerfile for python app.

# Use an official Python runtime as a parent image

FROM python:2.7-slim

# Set the working directory to /app

WORKDIR /app

# Copy the current directory contents into the container at /app

ADD . /app

# Install any needed packages specified in requirements.txt

RUN pip install -r requirements.txt

# Make port 80 available to the world outside this container

EXPOSE 8000:8000

CMD ["python", "app.py"]

==> Create app.py

from flask import Flask

import mysql.connector as mariadb

def fnconnDB():

myhost = os.environ.get('MYSQL_LOCAL_IMAGE_PORT_3306_TCP_ADDR')  #this will get the db server's IP which need to link during the docker run command. see the below docker run commnd

mariadb_connection = mariadb.connect(host=myhost,user='xxx', password='xxx',database='xxx')

return mariadb_connection

#......your codes….

#

if __name__=="__main__":

    app.run(host='0.0.0.0',port=int('8000'),debug=True,threaded=True)

==> Build the image

#docker build -t mysql-python-image .

==> run the conatiner with --link option, which will set mysql environment on python app container.

Syntax => docker run -d -p 8000:8000 --link <mysql container name>:<mysqk container image name> --name mysql-python-app mysql-python-image

#docker run -d -p 8000:8000 --link mysql-local-server:mysql-local-image --name mysql-python-app mysql-python-image

opened ports on linux using python

#Python script for checking the list of opened ports in linux...

import socket,datetime
ip_list = open("ip_list.txt").readlines()
port_list = open("port_list.txt").readlines()
open_ports = []
failed_ports = []
def pscan(ips,ports):
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.settimeout(2)
    try:
        con = s.connect((ips,ports))
        return True
    except:
        return False
print ('This Scan started on'), (datetime.datetime.utcnow() - datetime.timedelta(hours=5))
for ips in ip_list:
    for ports in port_list:
        con = pscan(str(ips.strip()),int(ports.strip()))
        if con == True:
            open_ports.append(ports.strip())
        else:
            failed_ports.append(ports.strip())
    print u"Device IP - " + ips.strip(), "Allowed", u"\u001b[32m" ,open_ports, u"\u001b[37m","Denied", u"\u001b[31m", failed_ports, u"\u001b[37m"
    open_ports = []
    failed_ports = []
print ('This Scan Ended on'),(datetime.datetime.utcnow() - datetime.timedelta(hours=5))

Open Source TIPs.

==> cat file1
==> tac file1 -> This will cat the file in reverse order.

==> Tips to use previous commands arguments...
      # mkdir /tmp/xyz  --> create the a directory on /tmp/xyz
      # cd !$           --> will redirect the directory directly.
==> screen--
      screen -S name
      screen -dr name
      screen -list
==> find --
      find / -xdev -type f -size +1G -exec ls -lh {} \;     --> find a file learger than 1GB.

==> echo -e "password\npassword" | passwd testuser          --> will set the password for test user.

==> change the history size -- Add the below in ~/.bashrc, Ex- if you want to change the file size into 40k
      HISTSIZE=40000
      HISTFILESIZE=40000
==> Display the history command with date and time in Linux.. using below..
    Define an environment varibale as below..
      #HISTTIMEFORMAT="%d/%m/%y %T"
    or
      echo 'export HISTTIMEFORMAT="%d/%m/%y %T"' >> ~/.bash_profile
    Here - %d is the day, %m is the month, %y is the year, %T is the time.
==> Erase duplicate entry on history.
      Define an environment varibale as below..
        #export HISTCONTROL=erasedups
==> View history without line numbers
      # history | cut -c 8-
      or
      # history | awk {'print $2'}

==> Record the command session--(using script command)
      # script file1.txt
      Execute your commands..and type exit once you done with the commands and want to save those.\
      # exit
      #cat file1.txt
==> make file immutable -
      touch file1
      chattr +i file1  -> (see the attribute using lsattr)

==> Process running related on Linux..
      The Kernel creates directory to each process on /proc with the pid.
==> snooppy is tool/app in linux can be used to track the executed commands.

==> Copy all files excluding certain files..
      #shopt -s extglob
      #cp !(*.mp3|*.txt) /tmp/
      # The above command will only copy the files extension with mp3 or txt
==> Copy a file to multiple directories..
      # echo /home/user1 /home/user2 | xargs -n 1 cp -v /tmp/test.txt
==> To stop all the running containers using docker.
      # docker stop $(docker ps -q)
==> curl command tips
      # curl -k <url>   --> will ignore the ssl certificate.
      # curl -u username:password <url>   -->(if a website asks a password)
      # curl -L <url>   --> will automatically redirect to new url.
      # curl -O <url>   --> will save the file in same name.
      # curl -o <url>  <file-name>  --> if small o is there it is require to give the file name.
==> Get your IP address
      # myip=$(curl --silent ipecho.net/plain)
      # echo $myip   --> will show your IP
==> Tips on how to use previous commands
      # up arrow & enter     --> will execute the previous command.
      # Type '!!' & enter    --> will execute the previous command.
      # Type '!-1' & enter   --> will execute the previous command.
      # Ctrl + P  (Equivalent to up arrow)  --> will execute the previous command.
      # Ctrl + n            --> Equivalent to down arrow
      # !mo:p               --> will print the previous command starteing with "mo"
==> Viewing files without comments and blank lines.
      # egrep -v "^#|^$" <config file name>
==> Terminal Tips..
      # Ctrl + w  --> Delete one word before cursor.
      # Ctrl + u  --> Delete all left side
      # Ctrl + K  --> Delete all right side.
      # Ctrl + y  --> undo the above
      # Ctrl + a  --> Moves to beginning of the line.
      # Ctrl + e  --> Moves to end of the line.
      # Ctrl + t  --> Swaps the character with left one.

Top 5 high memory or cpu consuming process

ps -eo pmem,pid,cmd | sort -k 1 -nr | head -5     ---> To list out the top 5 memory consuming processes
ps -eo pcpu,pid,cmd | sort -k 1 -nr | head -5     ---> To list out the top 5 CPU consuming processes

sample drone yaml example

    Drone is a CI/CD tool which is integrated with git repo and executes automatically for testing the pipeline from git repo when an update/push happened from git.

 * It has multiple sections and each one execute one after another inside the drone.
In below example, "clone", "build", "env_set",  "publish" are multiple sections. Inside each section the require parameters are there which works individually for that section only.

 * save as ".drone.yaml" in the repo and add to git repo.

 * Allow the git to execute from drone end.

--------------------------------------  xxxxxxxxxx   --------------------------------------------------------------
pipeline: # (started from here for pipeline)

  git_clone_section_here:  # (section for clone the git to drone, any name is fine)
    image: plugins/git
    recursive: true
    tags: true
    submodule-update-remote: true

  build:  # (section for build the image)
      image: "plugins/docker:latest"
      dry_run: true
      repo: <docker hub url>
      registry: <docker reg, name>
      dockerfile: .dockerfile
      when:
        branch:
          exclude: master


 env_set: #
      image: "<image name>"
      commands:
          - env
      when:
        event: push
        branch:
          exclude: master  # exclude the master branch



publish: (section for publish the image)
      image: "plugins/docker:latest"
      repo: <docker hub url>
      registry: <docker reg. url>
      docker.registry: <docker reg. url>
      dockerfile: .dockerfile
      environment:
      - PLUGIN_ENV_FILE=/drone/docker-env
      when:
        event: tag # drone execute when git tag / git version chages