Search

Fernando Partida's Blog

Software Quality and Testing

Month

April 2019

DevOps part 4, JUnit(or other) Automation (Individual)

This DevOps assignment is probably the most compicated of them all as it was a culmination of everything done in the previous DevOps. First of all I used pyTest as my unit testing tool as it was the easiest to install and use in my virtual machine. Then for all the gitHub stuff I used my pytesting-Quality repository.

First I created a script in python that creates a local webserver that reads the index.html file located at my scripts root folder. This file is constanly overwritten, and the code for the server is the following:

import http.server
import socketserver

PORT = 8080
Handler = http.server.SimpleHTTPRequestHandler

with socketserver.TCPServer(("", PORT), Handler) as httpd:
    httpd.serve_forever()

Then I created another python string in charge of testing the pytest_test.py file and seeing if it throws a “.” if the test succeeds or “F” if it fails. Then it updates the index.html file, with the status of the test. After doing so, the program updates the README.md file according to the status. Finally the script adds, commits with message and pushes the pytest_test.py and README.md files. The following is the code for doing so:

import subprocess

c = subprocess.check_output('pytest ~/PycharmProjects/pytest_test/pytest_test.py |grep "pytest_test.py "', shell=True)

s = c.split(b' ')[1].decode("utf-8")

if (s=="."):
        subprocess.call('echo "pytest test passed" > index.html', shell=True)
        subprocess.call('echo "# pytest test passed" > ~/PycharmProjects/pytest_test/README.md', shell=True)
        subprocess.call('git -C ~/PycharmProjects/pytest_test add pytest_test.py',shell=True)
        subprocess.call('git -C ~/PycharmProjects/pytest_test commit -m "pytest_test updated"', shell=True)
        subprocess.call('git -C ~/PycharmProjects/pytest_test add README.md', shell=True)
        subprocess.call('git -C ~/PycharmProjects/pytest_test commit -m "README.md updated"', shell=True)
        subprocess.call('git -C ~/PycharmProjects/pytest_test push origin master', shell=True)
else :
        subprocess.call('echo "pytest test failed" > index.html', shell=True)
        subprocess.call('echo "# pytest test failed" > ~/PycharmProjects/pytest_test/README.md', shell=True)
        subprocess.call('git -C ~/PycharmProjects/pytest_test add pytest_test.py',shell=True)
        subprocess.call('git -C ~/PycharmProjects/pytest_test commit -m "pytest_test updated"', shell=True)
        subprocess.call('git -C ~/PycharmProjects/pytest_test add README.md', shell=True)
        subprocess.call('git -C ~/PycharmProjects/pytest_test commit -m "README.md updated"', shell=True)
        subprocess.call('git -C ~/PycharmProjects/pytest_test push origin master', shell=True)

Now finally for the pièce de résistance the crontab file was updated so that it runs the server continuously, and the tester on every pc reeboot. The following is teh code obtained from running the sudo crontab -l command.

# Edit this file to introduce tasks to be run by cron.
# 
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
# 
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').# 
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
# 
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
# 
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
# 
# For more information see the manual pages of crontab(5) and cron(8)
# 
# m h  dom mon dow   command
* * * * * /usr/bin/python3 /home/ferpart/Documents/Scripts/server.py
@reboot /usr/bin/python3 /home/ferpart/Documents/Scripts/gitPusher.py

The following is a GIF showing everything working. (Obviously not using crontab at the moment, as it wouldn’t update that quickly).

ezgif-4-b4197cd188c1
Advertisements

Week 9 Practical – Python Unit Testing (Individual)

For the first part of this assignment we were tasked with reading Simple Smalltalk Testing: With Patterns and posting a Hypothes.is annotation and the following is evidence of that.

Annotation 2019-04-08 130213

Now for the WayBackMachine it is a website that allows the user to see versions of a website for the past basically. This allows us to see how something used to look some years ago. For the following I used the Amazon website from the year 2006 and this was the result:

Annotation 2019-04-08 131002

We were also assigned for doing a course on Unit Testing and Test Driven Development in Python the following is evidence of pytest running on pyCharm:

Annotation 2019-04-08 135054

The course was a bit longer than I had expected, and got bored at many times, but it did allow me to learn how to start using unit testing for something different than java (in this case python). Something I did find quite interesting was the section about best practices. That is because learning how to do testing in the most “correct” way helps others and yourself in the future understand what you’re doing.

DevOps part 3, GitHub, SSH and keys (Individual)

For this week we were assigned to use 2-factor authentication and ssh for our github profiles. Fortunately or unfortunately (in context of this practice) I had already done the previous, so not much can be written about the previous task.I will of course insert an image to prove the previous.

Untitled

As for the automation i will be automating the pull of the following repository:

Advanced Programming GitHub

This is the code used for the automatic pull (note that the server automation used for the previous DevOps assignment has been removed)

# Edit this file to introduce tasks to be run by cron.                          
#                                                                               
# Each task to run has to be defined through a single line                      
# indicating with different fields when the task will be run                    
# and what command to run for the task                                          
#                                                                               
# To define the time you can provide concrete values for                        
# minute (m), hour (h), day of month (dom), month (mon),                        
# and day of week (dow) or use '*' in these fields (for 'any').#                
# Notice that tasks will be started based on the cron's system                  
# daemon's notion of time and timezones.                                        
#                                                                               
# Output of the crontab jobs (including errors) is sent through                 
# email to the user the crontab file belongs to (unless redirected).            
#                                                                               
# For example, you can run a backup of all your user accounts                   
# at 5 a.m every week with:                                                     
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/                               
#                                                                               
# For more information see the manual pages of crontab(5) and cron(8)           
#                                                                               
# m h  dom mon dow   command                                                    
0 18 * 1-6 1 /usr/bin/git pull origin master /home/ferpart/Documents/Code/gitClones/gitAPobed

The previous script that was added to the crontab, will pull from the aforementioned GitHub repository every Tuesday at 7:00 pm (or 19:00) for the rest of the semester which is the time at which the class is taken every week. A thing to note is that i wrote the time as 18 instead of 19 because the time value starts at 0.

DevOps part 2, Linux Server Setup (Individual)

For part 2 of DevOps we were assigned to install a linux distribution, run a server, and use crontab to schedule execution.

The linux distribution that i chose to use is that of Ubuntu 18.10 because i already had it installed as a virtual machine.

29985a98-ubuntu-logo32

For setting up the server I installed nodejs with the following command:

sudo apt install nodejs

With nodejs installed, a script for running the server was created at a folder designated for testing. The script written was the following:

const http = require("http");
const hostname = "127.0.0.1";
const port = 8080;

const server = http.createServer((req, res) = >{
        res.statusCode = 200;
        res.setHeader("Content-Type", "text/plain");
        res.end("Server test\n");
});
server.listen(port, hostname, () = > {
        console.log("El servidor esta corriendo");
});

For testing the following bash command can be run:

node test_server.js

This will show us the following screen:

Untitled

After that it was just question of sheduling it to run. For now i set it for running 24/7, 365 for testing with the following code:

First to enter crontab the following command was run:

sudo crontab -e

Then the script used was the following

# Edit this file to introduce tasks to be run by cron.                          
#                                                                               
# Each task to run has to be defined through a single line                      
# indicating with different fields when the task will be run                    
# and what command to run for the task                                          
#                                                                               
# To define the time you can provide concrete values for                        
# minute (m), hour (h), day of month (dom), month (mon),                        
# and day of week (dow) or use '*' in these fields (for 'any').#                
# Notice that tasks will be started based on the cron's system                  
# daemon's notion of time and timezones.                                        
#                                                                               
# Output of the crontab jobs (including errors) is sent through                 
# email to the user the crontab file belongs to (unless redirected).            
#                                                                               
# For example, you can run a backup of all your user accounts                   
# at 5 a.m every week with:                                                     
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/                               
#                                                                               
# For more information see the manual pages of crontab(5) and cron(8)           
#                                                                               
# m h  dom mon dow   command                                                    
* * * * * /usr/bin/node /home/ferpart/Documents/Code/DevOps/test_server.js

Asterisks were used so that it runns all the time.

Intro to DevOps (Individual plus peer review)

DevOps as its name states is short for “Development Operations”. This is a term that started from the combination of the agile operations and that which is the collaboration between the development and operation staff on all stages of the development cycle.

In general its described as stated before “it is a collaboration of the developer and operations”. The following are the generally known levels of a DevOps structure.

  • DevOps Values: These are fundamentally the same as those seen within the agile methodology. Just with slight tweaks focused on the service delivered to the customer.
  • DevOps Principles: One of the most commonly cited principles is that of “Infrastructure as code” otherwise, no other general principles have been agreed collectively upon.
  • DevOps Methods: Methods for implementation are generally the same as with other methodologies. these could be “scrum”, “kanban”, etc…
  • DevOps Practices: The most important practices is that of continuous integration and continuous development.
  • DevOps Tools: Tools that could be used are vast, but these are some examples; “configuration management”, “orchestration”, “monitoring”, “virtualization and containerization”, etc…

In general we can then understand that even though DevOps is a very good, versatile and easy to use tool, It is not very easily defined.

 

Create a free website or blog at WordPress.com.

Up ↑