Tag: python

April 15, 2018April 14, 2018

RS232 sniffing with Python

Published on: 15.04.2018

My motivation for monitoring RS232 with Python.

I had PC and LG TV screen communicating via RS232 cable.

PC was turning TV screen on and off via RS232 commands.

But TV screen was not acting normally, it was always in standby mode.

So, either PC was not sending RS232 commands or TV screen was not responding to them.

In order to diagnose the problem, I decided to spy on their communication.

half-duplex and full-duplex RS232

There are two types of communication in RS232: half-duplex and full-duplex.

In half-duplex while one device is talking another one is listening, just like people communication should be.

In full-duplex both devices are talking and listening at the same time, people think that they can do this also, but it is not working in practice.

Hardware

The first step was to make special half duplex RS232 spy cable because I did not have software access on PC.

So the only solution was to sniff communication between PC and TV.

Schematics are available from https://www.lammertbies.nl/comm/cable/RS-232-spy-monitor.html.

On the same website, there are also Schematics for full duplex RS232 spy cable.

Software

For redirecting, all communication data from the RS232 compliant serial port device into a text file Eltima RS232 Data Logger can be used.

The program is working fine, the only problem that I had was that there is no output on screen just to text file.

No real-time watching of communication data was possible, due to that I decided to make my own program for viewing RS232 communication data in real-time.

Python to the rescue

The best thing about Python program language is that there is a package for everything.

For access to the serial port in Python, there is pySerial package.

My Python program was reading communication data and showing it on screen in real-time with timestamps of reading, and time difference between the last read.

Python source code for RS232 sniffing:

import serial    # pyserial
import datetime


def to_hex(input_str):
    outpur_str = " ".join("{:02x}".format(ord(c)) for c in input_str)
    # 4b 41 20 30 20 46 46 0d
    return outpur_str


def main():
    ser = serial.Serial(
    port='COM1',
    baudrate=9600,
    parity=serial.PARITY_NONE,
    stopbits=serial.STOPBITS_ONE,
    bytesize=serial.EIGHTBITS
    )

    # open if not opened
    if ser.isOpen() == False:
        print "OPENING port."
        ser.open()

    # header
    fmt = "{0:15}{1:35}{2}"
    print fmt.format('TIME', 'RAW HEX', 'TIME DELTA')

    # so to have first value
    time_of_last_read = datetime.datetime.now()
    
    while True:
        data_to_read = ser.in_waiting
        
        if data_to_read != 0: # read if tehre is new data
            time_now = datetime.datetime.now()   
            
            data = ser.read(data_to_read)
            hex_with_space = to_hex(data)
            time_with_miliseconds = time_now.strftime("%H:%M:%S.%f")[:-3]

            time_delta = time_now - time_of_last_read
           
            print fmt.format(time_with_miliseconds, hex_with_space, time_delta)

            # last time
            time_of_last_read = time_now
 
    ser.close()
    
if __name__ == '__main__':
    main()

import serial # pyserial

import datetime

def to_hex(input_str):

outpur_str = " ".join("{:02x}".format(ord(c)) for c in input_str)

# 4b 41 20 30 20 46 46 0d

return outpur_str

def main():

ser = serial.Serial(

port='COM1',

baudrate=9600,

parity=serial.PARITY_NONE,

stopbits=serial.STOPBITS_ONE,

bytesize=serial.EIGHTBITS

)

# open if not opened

if ser.isOpen() == False:

print "OPENING port."

ser.open()

# header

fmt = "{0:15}{1:35}{2}"

print fmt.format('TIME', 'RAW HEX', 'TIME DELTA')

# so to have first value

time_of_last_read = datetime.datetime.now()

while True:

data_to_read = ser.in_waiting

if data_to_read != 0: # read if tehre is new data

time_now = datetime.datetime.now()

data = ser.read(data_to_read)

hex_with_space = to_hex(data)

time_with_miliseconds = time_now.strftime("%H:%M:%S.%f")[:-3]

time_delta = time_now - time_of_last_read

print fmt.format(time_with_miliseconds, hex_with_space, time_delta)

# last time

time_of_last_read = time_now

ser.close()

if __name__ == '__main__':

main()

April 1, 2018November 23, 2017

Fluent Python, book review

Fluent_Python-Book_Cover

Published on: 01.04.2018

Number of pages: 792
Written by: Luciano Ramalho
Publish by: O’Reilly Media

Conclusion
The book is good for advanced Python, it covers a lot of and in details.

Review
This book is for developers who want to get their Python skills to advance level.

It is not for beginners, for beginners in Python I recommend Automate The Boring Stuff With Python.

Book Effective Python also covers advanced topics, but this one covers in more details.

Both books are good, I would recommend this one because it describes topics in more details, but that is why this book has much more pages.

March 15, 2018February 20, 2018

How to find all emails on the web page ?

Published on: 15.03.2018

Conclusion

Use get_emails() from webscraping Python package.

Python strength

The best thing about Python is huge numbers of 3rd party packages.

With a lot of them, you can solve your problems with just a few lines of code.

Let’s say that you want to find all emails in some HTML document, either for an offline or online web page.

This can be done with webscraping package.

First, install it with:

pip install webscraping

1	pip install webscraping

Code for finding all emails on the single page is:

from webscraping import download, alg

D = download.Download()
html = D.get('http://buklijas.info/')

emails = alg.extract_emails(html)

print emails

from webscraping import download, alg

D = download.Download()

html = D.get('http://buklijas.info/')

emails = alg.extract_emails(html)

print emails

Line 1 is importing download and alg from webscraping package that you have just installed.

Line 3 is creating download.Download() object and calling it D.

Line 4 is saving the web page from where you want to find all emails in html variable.

Line 6 is finding all emails from your html variable and saving all emails in emails Python list.

Line 8 is showing all emails that have been found on the screen.

This will work for a single web page.

How to find emails on the whole site

If you want to search the whole website for emails, not just one page, you can use following code.

from webscraping import download

D = download.Download()

emails = D.get_emails("http://buklijas.info/", max_depth=2, max_urls=None, max_emails=None)

print emails

from webscraping import download

D = download.Download()

emails = D.get_emails("http://buklijas.info/", max_depth=2, max_urls=None, max_emails=None)

print emails

With max_depth, max_urls, max_emails parameters you can define how long your searching should be.

Happy spamming.

P.S. just joking 🙂

February 15, 2018March 10, 2018

Should you learn C/C++ in 2018 ?

Published on: 15.02.2018

Conclusion

I would not recommend learning C/C++ as your first programing language just for learning to programme, better go with Python.

Also, if you do not know why you want to learn C/C++, better do not spend time on it.

My C/C++ background

Most of my professional programming career, around 10 years I have spent writing C/C++ programs.

By professional programming career, I mean that other people have paid me to write code.

I have spent more time in C/C++, altho I would say that I know Python better than C/C++.

My knowledge of Python is better because Python has fewer features than C/C++ and you do not need to do memory management.

Why C/C++ is hard

IMHO C/C++ is hard due to many features and memory management.

By the time you understand memory management, you can learn 80% of Python.

Just look at this image of code that is concentrating 2 strings in C vs. Python.

Yes, there can be less C code (error checking, free, few const less) and the code would still work.

But then I would be showing bad C production code and not be demonstrating how real-world C code should be written.

It is not fun to debug segmentation fault and that is what you get if there is no proper error checking in C.

Python code could, and in real life should also have if __name__ == "__main__":, but I have removed it to because it is not necessary like in C.

Point is that higher level programing language (Python in this example) is doing a lot of low-level things automatically.

So, you need fewer lines of code, what means fewer possible bugs, resulting in increased developer productivity.

The tradeoff is more CPU and memory for faster time to market.

C and C++ are two different programming languages

Here I am using a language construct C/C++.

Because most of the code bases (at least that I have seen in a corporate environment) are some hybrid of C and C++.

My theory is that the older code was written in C and then later they added object-oriented programming (C++).

But C and C++ are not the same programming languages, mental programming models are quite different.

C code is about structure, functions, and pointers.

In C++ you have object-oriented programming and lot of other features.

In the real world, you need to know C and C++, so that is why I use C/C++.

When you should learn C/C++

The only positive thing that I can see that person will learn from learning C/C++ is manual memory management and pointers.

Basically better understanding how the computer (on software level) is working.

If your only reason is to have a better understanding how the computer (on software level) is working, better learn C programing language.

C has fewer features than C++.

It is easier to learn C++ if you know C because then you should know manual memory management and pointers.

But for anything else than few niches (system, embedded, banking, game engines, etc), I do not see a much practical use of C/C++ in the year 2018.

Probably you ain’t gonna use C/C++

I am not saying C/C++ is not in use anywhere in 2018.

I personally (almost every month) get some interview requests for C/C++ positions.

Usually, they are either for embedded software (C) or banking industries (C++).

I am just arguing that beginners (persons that do not know any programing language) should not start with C/C++.

The only exception is if you are planning to get a job (or start your business) in C/C++ environment.

If it is not for the job/business, I do not see why you should learn C/C++, except for hobby.

But even then, get a better hobby 🙂.

P.S. If you, dear reader, think that I am missing some point, please add it in comments.

January 1, 2018December 31, 2017

Always start with simple solution (ConfigParser vs. JSON for Python configuration file)

Published on: 01.01.2018

Conclusion

Sometimes perfect is the enemy of the good.

Why use configuration file

I had Python program, that needed to access device via IP address and to do some diagnostic and commands on it.

At that time, I was accessing only one device (only one IP address).

But could see that in future (in few months to one year), I will need to do the same set of command on more devices.

One solution is to add IP address as a parameter to CLI program.

In my use case number of IP address that needs to be accessed will never be bigger than 34.

And writing 34 IP addresses as CLI parameter, that is around 373 letters, is not a nice solution.

When you need to read it, to see are all IP addressee included, it is not easy to read.

Python code as configuration was not possible

I was distributing my Python code as EXE, so use of Python code as configuration was not possible.

Altho, I think that Python code as the configuration is a good solution if you are executing source code, and only developer (not average user who does not know what Notepad is) will edit it.

Sometimes perfect is enemy of good

Sometimes I do have a problem that I tend to unnecessarily complicate things.

Because I think about all possible edge cases and all possible future uses.

And from these two, all possible future uses are the real problem.

Edge cases can happen, but is there a positive cost benefit to solve them and how?

This need to be determined on a case by case basis.

But “all possible future use” is trying to anticipate the future.

That is impossible.

From my experience, whenever I add code for future use cases, usually it was waste of time.

Even when I am the only user, so I can argue to my self that I know will need it.

Usually, I do not need it.

So, I have decided to eliminate waste when developing software, starting from this project.

For example, here I needed a configuration file that will have a list of IP addresses, which I will iterate in for loop.

This was the smallest requirement that I needed for my problem.

But immediately I was thinking that it would be nice to have:

checking if IP address is in the valid format
if I have 5 address that I need to write all 5, but I could add special syntax for that. Like 192.168.1.1 --- 192.168.1.5, and this can really get more complicated if you want to cover all edge cases.
I only need IPv4, but can I also ad IPv6
if there is some error in the configuration file, there can be a useful error message and suggested a solution
another configuration file can be added where I would define what are valid IP addresses, and then I can validate requested IP against valid IP

And the list can go on.

But I said to myself, NO.

You will just make code that can read a list of items as a list in Python from the configuration file.

Nothing more.

If when you have the real need, you will add additional features when needed.

When I think about this, maybe this is an example of premature design?

Premature design is deciding too early what a program should do.

Why JSON is better than Configparser for list datatype

After investigating possible solution, decision was to try ConfigParser.

ConfigParser configuration file was:

[other]
list_queue1 = [ "192.168.1.34", "192.168.1.35",
               192.168.1.36,
               "192.168.1.34" ]

[other]

list_queue1 = [ "192.168.1.34", "192.168.1.35",

192.168.1.36,

"192.168.1.34" ]

Code was:

import ConfigParser
import io

def get_as_list(text):
    _ = text.replace('[', '')
    _ = _.replace(']', '')
    _ = _.replace('"', '')
    tmp_list = [ item.strip() for item in _.split(',') ]
    return tmp_list

def main():
    # Load the configuration file
    with open("config_ini.ini") as f:
        sample_config = f.read()
    config = ConfigParser.RawConfigParser(allow_no_value=True)
    config.readfp(io.BytesIO(sample_config))

    # access them
    testing_1 = config.get('other', 'list_queue1')
    for item in get_as_list(testing_1):
        print item

if __name__ == '__main__':
    main()

import ConfigParser

import io

def get_as_list(text):

_ = text.replace('[', '')

_ = _.replace(']', '')

_ = _.replace('"', '')

tmp_list = [ item.strip() for item in _.split(',') ]

return tmp_list

def main():

# Load the configuration file

with open("config_ini.ini") as f:

sample_config = f.read()

config = ConfigParser.RawConfigParser(allow_no_value=True)

config.readfp(io.BytesIO(sample_config))

# access them

testing_1 = config.get('other', 'list_queue1')

for item in get_as_list(testing_1):

print item

if __name__ == '__main__':

main()

I found few problems with ConfigParser:

It did not have build in functionality to read data as Python List
- so even for this simple example I had to write custom code (this is why I have get_as_list() function)
There were no rules in format, eg. "192.168.1.36" and 192.168.1.36 was both valid.
- this was not big problem, but I just did not like it

Let try JSON (as configuration files in Python)

JSON configuration file was:

{
    "test_list":[
        "test_1",
        "test_2",
        "test_3"
    ]
}

{

"test_list":[

"test_1",

"test_2",

"test_3"

]

}

Code was:

import json

def main():
    with open('config_json.json') as json_data_file:
            data = json.load(json_data_file)

    for item in data['test_list']:
        print item

if __name__ == '__main__':
    main()

import json

def main():

with open('config_json.json') as json_data_file:

data = json.load(json_data_file)

for item in data['test_list']:

print item

if __name__ == '__main__':

main()

There is much less code, and line count is important.

One thing that I did not like about JSON is "ValueError: No JSON object could be decoded", this is error message that you get if you JSON is invalid.

I was hoping to get some more details, eg. what token in which line.

But you can not have it all, and ConfigParser was no bettter.

Decision was to use JSON as configuration file for Python, because code was less complicated (less lines of code).

I did not tried YAML.

Syntax, at least to me, is less readable than JSON.

Questions and remarks please leave in comments.

December 1, 2017November 23, 2017

Automatic backup of git repositories to Dropbox with Python

Published on: 01.12.2017

Intro

I will show how to upload files to Dropbox from Python code.

Why do I need this?

Currently, I am only using WebFaction for all my web services and also as my private git server.

I wanted to make an automatic backup of my git repositories to Dropbox.

Dropbox App

I order to upload files to Dropbox you need to have an access token.

And for the access token, you need to register your app on DBX platform.

All of this must be done on Dropbox website.

The first step is to go to https://www.dropbox.com/developers/apps/ and press “Create App” button.

Step 1

Just click

Just click “Create app” button

Step 2

New app on DBX Platform

We will use Dropbox API.

We will choose “App folder” because we will just upload one backup to Dropbox, we do not need full access to all our files.

Name your app and click “Create App” button.

Step 3

Use defaults settings

We will use defaults settings, here we will get the access token so, click “Generate access token” button.

Step 4

Access token generated

Now you have your access token, you will need it in your code, so copy it.

Step 5

Dropbox app

Now we have our “my_git_backup” Dropbox app.

pip install

pip install dropbox
pip install fabric

1 2	pip install dropbox pip install fabric

It is always recommended to use virtual environments inside python.

At least I use them always.

Code

from fabric.api import local, lcd
import datetime
import dropbox

remote_directory = 'full path to root git repository'
NUMBER_OF_BACKUP_TO_KEEP = 10

from fabric.api import local, lcd

import datetime

import dropbox

remote_directory = 'full path to root git repository'

NUMBER_OF_BACKUP_TO_KEEP = 10

I am using fabric to make my life(code) easier.

I use fabric every time when I am calling CLI command from Python.

I will explain NUMBER_OF_BACKUP_TO_KEEP later, I use it at the end of the program.

def git_backup_to_dropbox():
    with lcd(remote_directory):

1 2	def git_backup_to_dropbox(): with lcd(remote_directory):

All code that follows is inside with lcd(remote_directory): Python context manager.

The context manager is used so all code that follows is executed inside remote_directory directory.

name_of_backup = datetime.datetime.today().strftime('%Y-%m-%d_%H%M%S')
name_of_backup += '_git_backup.zip'

local('zip --password LAME_PASSWORD -r ' + name_of_backup + ' *.git')

name_of_backup = datetime.datetime.today().strftime('%Y-%m-%d_%H%M%S')

name_of_backup += '_git_backup.zip'

local('zip --password LAME_PASSWORD -r ' + name_of_backup + ' *.git')

Name of backup file will be YYYYMMDD_HHMM_git_backup.zip where upper case letters are date and time when a program was executed.

Eg. 20171121_1856_git_backup.zip so that we know from when is this backup file.

For making an actual backup, zip CLI command is used, we are only doing the backup of files that end on *.git (in my case only git repositories).

I also have LAME_PASSWORD for basic protection.

This is why I used fabric, just by calling local() function you can execute CLI commands.

dbx = dropbox.Dropbox('YOUR ACCESS TOKEN FROM DROPBOX!!!')

with open(remote_directory + '/' + name_of_backup, 'rb') as f:
    dbx.files_upload(f.read(), '/' + name_of_backup )

local('rm ' + name_of_backup)

dbx = dropbox.Dropbox('YOUR ACCESS TOKEN FROM DROPBOX!!!')

with open(remote_directory + '/' + name_of_backup, 'rb') as f:

dbx.files_upload(f.read(), '/' + name_of_backup )

local('rm ' + name_of_backup)

The first line is the opening connection to your Dropbox application, you need to add your own access token as an argument.

Next two lines are for upload, you are: opening file, reading it and uploading bytes to Dropbox.

In Dropbox documentation is mention that this is only working for files till 150MB in size.

With last line program is deleting the local backup.

all_backup_files = []
for entry in dbx.files_list_folder('').entries:
    all_backup_files.append(entry.name)

for file in sorted(all_backup_files[:-NUMBER_OF_BACKUP_TO_KEEP]):
    dbx.files_delete('/' + file)

all_backup_files = []

for entry in dbx.files_list_folder('').entries:

all_backup_files.append(entry.name)

for file in sorted(all_backup_files[:-NUMBER_OF_BACKUP_TO_KEEP]):

dbx.files_delete('/' + file)

First for loop is getting all files from your backup folder in a list.

Second for loop is deleting all files except, last few files.

How many files to keep (otherwise we need manually to delete old backup files) is define in NUMBER_OF_BACKUP_TO_KEEP from the beginning of the code.

I keep it at 10, more than that I do not need.

Because we have date and time in our filename we can use Python sort function to sort files by when the backup was done.

The program can be run with
fab -f fabfile git_backup_to_dropbox

First is fab because we used fabric, fabfile because fabfile.py is file of our source code and git_backup_to_dropboxis name of the function that we are executing from fabfile.py file.

How I run this automaticaly

I personally run this command from crontab once per day.
35 02 * * * /home/user_name/code/venv/bin/fab -f /home/user_name/code/fabfile git_backup_to_dropbox

Conclusion

This can be used for backup of any folder as zip file automaticaly to Dropbox.

For any questions, please write them in comments.

November 1, 2017December 15, 2018

Controlling NEC display from Python with nec-pd-sdk

Update on: 07.03.2018
Updated version available on Twilio blog.

Published on: 01.11.2017

Conclusion

pip install nec-pd-sdk

My Story

I was responsible for maintenance of one spectacular 17 meters tall audio/video system on a cruise ship.

The system had 34 NEC X551UN screens among other components.

Waterfall from top

Behind each screen, there is a SDI-to-DVI converter.

If a picture on the screen was black, usually there was some problem with SDI-to-DVI converter, mostly power supply was broken.

Or NEC screen was broken, but I never had it in practice.

Special NEC screen

But, also there was one special screen, it was black from time to time.

After restart mostly fine, and SDI-to-DVI converter was fine.

After one month of troubleshooting, I have come to the conclusion that problem is with NEC screen.

It just got stuck every few days (sometimes every second day, some time was fine for a week), and simple restart (sometimes of 5 seconds and sometimes of 5 minutes) would solve issue till next time.

I also know that when there is a black picture on this screen, then screen diagnostic was “No signal.”

NEC screen no signal

I have come to the conclusion that the following code could solve the problem:

if diagnostic_of_screen is "no_signal" is True;
    turn screen OFF
    wait 5 minutes
    turn screen ON

if diagnostic_of_screen is "no_signal" is True;

turn screen OFF

wait 5 minutes

turn screen ON

And then to run this code on a schedule, like every hour.

Existing NEC software

NEC have two software applications for managing their products.

First is PD Comms Tool, you can remotely get and set all values to a screen.

It also has a scripting language.

I have used it for setting scheduler for all 34 screens and change of time.

It is much faster than manually doing it for each screen.

The second one is NaViSet Administrator 2 it is much more powerful than “PD Comms Tool”.

It can be used for monitoring all your NEC screens and also some additional equipment (like projectors and Windows PC)

It also has a visual scripting language where you can set and get multiple parameters according to some condition.

And then you also can set specific scheduler for each script.

I could have used this tool for my problem, but there was just one problem, it did not have sleep/pause command.

Design

I know that existing NEC software is communicating with the screen via TCP/IP.

Full protocol documentation is at http://www.necdisplay.com/documents/UserManuals/External_Control_P.V.X-series.pdf, but I was not so eager to write custom TCP/IP packets.

I wanted something more readable and simple.

I googled “NEC python” and found about nec-pd-sdk, what is python SDK for NEC screens.

There is no textual documentation, but there are few examples.

Most useful for me was test_routines_example.py, it is showing how to get every parameter.

The command for turning screen ON and OFF was found in source code.

Code for turning NEC screen ON and OFF

Here is the code:

from __future__ import print_function
from nec_pd_sdk.nec_pd_sdk import NECPD
from nec_pd_sdk.protocol import PDError
from nec_pd_sdk.constants import PD_POWER_STATES
import time  # for sleep
import sys


def get_screen_state(pd):
    state = pd.command_power_status_read()
    text_list = pd.helper_self_diagnosis_status_text()
    return {'status': state, 'diagnosis': text_list}


def main():
    try:
        pd = NECPD.open('192.168.1.52')

        monitor_id = 1
        pd.helper_set_destination_monitor_id(monitor_id)

        try:
            screen_state = get_screen_state(pd)

            # 'Normal; '      'No signal; '
            if screen_state['diagnosis'] == 'No signal; ':
                
                # OFF
                pd.command_power_status_set(PD_POWER_STATES['Off'])

                # pause
                time.sleep(5 * 60)  # 5 minutes

                # ON
                pd.command_power_status_set(PD_POWER_STATES['On'])  # 1

        finally:
            # make sure to always close
            pd.close()

    except PDError as msg:
        print("PDError:", msg)
    return


if __name__ == '__main__':
    main()

# https://github.com/pyinstaller/pyinstaller/issues/1687
sys.exit()

from __future__ import print_function

from nec_pd_sdk.nec_pd_sdk import NECPD

from nec_pd_sdk.protocol import PDError

from nec_pd_sdk.constants import PD_POWER_STATES

import time # for sleep

import sys

def get_screen_state(pd):

state = pd.command_power_status_read()

text_list = pd.helper_self_diagnosis_status_text()

return {'status': state, 'diagnosis': text_list}

def main():

try:

pd = NECPD.open('192.168.1.52')

monitor_id = 1

pd.helper_set_destination_monitor_id(monitor_id)

try:

screen_state = get_screen_state(pd)

# 'Normal; ' 'No signal; '

if screen_state['diagnosis'] == 'No signal; ':

# OFF

pd.command_power_status_set(PD_POWER_STATES['Off'])

# pause

time.sleep(5 * 60) # 5 minutes

# ON

pd.command_power_status_set(PD_POWER_STATES['On']) # 1

finally:

# make sure to always close

pd.close()

except PDError as msg:

print("PDError:", msg)

return

if __name__ == '__main__':

main()

# https://github.com/pyinstaller/pyinstaller/issues/1687

sys.exit()

Last Words

When I was investigating NaViSet Administrator 2 for my use-case.

I contacted NEC support, to ask can them can I use it for this purpose.

They told me not and suggested to use TCP/IP External_Control.

So, even support from NEC does not know that they have NEC python SDK.

What is sad, considering that their NEC python SDK is useful software.

Questions and remarks, please leave in comments.

October 1, 2017September 11, 2017

Full Stack Python Guide to Deployments, book review

full_stack_python_guide_to_deployments-Book_Cover

Published on: 01.10.2017

Number of pages: 262 with code example, 185 without
Written by: Matthew Makai
Publish by: Self-publish

Conclusion
Useful to read, if you want to learn/see how to deploy python web application.

Review
Topic of book is described in its subject, full stack python deployments.

In appendix C of book is full source code of Flask application that you are deploying.

Topic of book is not source code from Flask application, but just how to deploy that code.

Deployment is done on virtual server on Linode (Ubuntu).

Deployment process is automated with Fabric and Ansible.

I liked this automation because you can use it for your own project also.

Other technologies used are: Nginx, Git, PostgreSQL, Redis, Green Unicorn, Celery and Jenkins

September 1, 2017July 23, 2017

Make standalone executable from Python code with PyInstaller

Published on: 01.09.2017

I wanted to create single file that person could run on Windows machine, from my Pythone code.

After some investigation I found PyInstaller and 1 hour later I had my EXE file from Python code.

Process for generating EXE files from Python code with PyInstaller was quite easy, at least from my experience.

I have used it on Windows 7 64-bit and had no problems.

My program was one file script with 300 line and dependencies to docopt and pyautogui.

Steps for generating exe file with PyInstaller

This will install PyInstaller
pip install pyinstaller

This will generate script.exe in dist directory
pyinstaller --onefile script.py

After this you have your EXE program.

How PyInstaller is working

Here I have used --onefile option for PyInstaller what will make one file EXE program.

If you just use pyinstaller script.py, with out --onefile option, than in dist folder you will get folder script with EXE file and all additional files for your EXE file to work.

If you use --onefile option, then your one file EXE program need every time to uncompresses all files every time when it starts.

Uncompression is described in details in official documentation, temporary directory for uncompression in Windows is %TEMP%.

Some other solutions:
http://nsis.sourceforge.net/Main_Page
http://nuitka.net/
https://pypi.python.org/pypi/pynsist
https://cx-freeze.readthedocs.io/en/latest/
http://www.py2exe.org/

Comparison of some others solutions.

July 1, 2017February 25, 2017

Grokking Algorithms, book review

Grokking_Algorithms-Book_Cover

Published on: 01.07.2017

Number of pages: 256
Written by: Aditya Bhargava
Publish by: Manning

Conclusion
Interesting book, visualizations are very helpful.

Review
When I saw this book first time I was not interested.
Book cover was too childish for me.

But after listening podcast about it, I changed my mind.

Illustrations in book are used to explain concepts, and they are very useful.
All code is in python, that was bonus point for me.

Here is video example of illustration:

I do recommend this book for reading, you can understand Big O notation from it and algorithms.