Introducing PyAutoGUI

I found out about PyAutoGUI Python packet from Automate the Boring Stuff with Python book.

With PyAutoGUI you can automate GUI interaction on your computer.

PyAutoGUI is working on Windows, OSX and Linux, altho I have used it only on Windows.

Most GUI automation can be done just by automating mouse movements and keyboard input and PyAutoGUI supports it.

There are also functions for providing message boxes, saving the screenshot and finding an element from the image on the screen.

Personally, I have found PyAutoGUI useful, for automating some of my workflows.

To be honest, I use it for 45 minutes of work every month (I know this is not much), but I have found that if I manually have to do same interactions for 45 minutes it is really killing my soul, so I decide just to automate it.

If you want to automate web page interaction use Selenium, because with Selenium you can access elements on a web page independent of their position on the screen.

Because with PyAutoGUI you can just move the mouse to specific coordinates, that means if the resolution or layout of GUI has been changed, you need to update coordinates in your code.

Keep all coordinate in your code as constant in one place, so that you do not need to change them all over your code when change is needed.

This theoretically could be avoided if you use images as reference for finding elements on your GUI, but you will have the same problem if the appearance of the elements is changed, but then instead of changing coordinate in the code you have to change all reference images, what is more work.

My personal preference is to use hardcoded coordinated instead of images as the reference.

Using Selenium has also become popular for scraping pieces of information from web pages, but better is to use specialized framework for scraping like Scrapy, because of additional features like caching HTTP responses (quite a time saver in development).

The Clean Coder: A Code of Conduct for Professional Programmers, book review

beginning_serverless_computing-Book_Cover

Published on: 15.10.2018

Number of pages: 211
Written by: Robert C. Martin (Uncle Bob)
Publish by: Prentice Hall

Conclusion
Read this book if you plan to make money by writing code for others.

Review
This is an interesting book.

It is not a soft skill book, but it is also not a technical book, it is something in between.

The book has valuable advices on how to behave as a professional programmer.

The author is writing software professionally since 1970, so there are a lot of stories about how the software was made back then. I liked those stories a lot.

Chapters titles are: Professionalism, Saying No, Saying Yes, Coding, Test Driven Development, Practicing, Acceptance Testing, Testing Strategies, Time Management, Estimation, Pressure, Collaboration, Teams and Projects, Mentoring, Apprenticeship and Craftsmanship, Tooling.

Making web apps with Jupyter notebook

This article will explain how to make Jupiter notebook as a GUI app on the web.

What is Jupiter notebook

Jupiter notebook is browser-based REPL.

REPL enables you to program in an interactive environment, you can write and then execute your next line of code while all previous lines are already in the executed state.

This trivial feature enables me to cut prototyping development time because for testing the next line of code I do not need to run the whole program again. (REPL is useful only for some types of situations)

I know this explanation is useless if you do not know what is a programming language and have no experience with REPL style prototyping, but if you are in this category I do not know how to explain it (probably it is impossible).

Point is, it makes programming prototyping faster because for testing next line in your code you do not need to run the previous code again and again.

Previously Jupiter notebook was called IPython Notebooks, at that time only Python was available as programing language.

Now it is possible to use Jupiter notebook with many programming languages, altho my experience is only with Python.

Personally, I use Jupiter notebook for exploratory data analysis.
Loading data to Pandas and then trying to understand data with visualizations (Seaborn, Bokeh).

Sharing Jupiter notebook with non-technical persons

Often I would run the same code with different parameters, to produce slightly different visualizations.

If you are familiar with Jupiter notebook environment than you know that this means running the same cell with SHIFT + ENTER, from Cell menu or some other shortcut.

This got me thinking if I wanted to give my notebook to a non-technical person (somebody who know how to use Word, and Excel without knowledge of how to write formulas ) it would be trivial for that person to use it.

Also, a person could change the code and get the unintended outcome (syntax error or wrong result).

Ipywidgets

This problem could be solved with Ipywidgets widgets.

With Ipywidgets widgets you can make GUI inside of Jupiter notebook, it is perfect when you want for somebody (even you) to expose some functionality of your Jupiter notebook with GUI elements.

For having this kind of GUI
Coin Toss GUI

This is the necessary code:

Appmode

This was good but still there where 2 problems:

  • the user had to run all cell as the first step
  • user still had access to the code

Fortunately, Appmode is Jupyter extensions that turn notebooks into web applications.

By default user can still go back in the “code mode”, but it can be easily removed

Hosting your Jupiter notebook

If you are hosting it inside of your network that you just need to run notebook server, like for local development, but add some security.

Github will give you view only of any notebook that is hosted on their server, and there are many more websites with the same functionality.

If you want interactive hosting of your Jupiter notebook so that people can execute them, then there is Binder.

Currently, it is in beta and your Jupiter notebook needs to be in public Github repository.

Conclusion

With the right combination of:

You can execute your Jupiter notebook as a web app for free.

Coin Toss code can be used as an example of how to host Jupiter notebook as a GUI app on the web.

Part of the inspiration came from Bloomberg bqplot project.

Personally, I have found it useful for sharing interactive visualizations.

Sending email from Python

I will show how to send emails from Python programing language using your Gmail account.

Mail server

When you want to send email from GUI/CLI app or from some computer language (in this example Python), you need to have access to a mail server.

The mail server is a computer that is in charge of sending and receiving your email.

Usually, you have access to the mail server via username and password.

With Gmail username is your email address and password should only be known to you :-).

Use yagmail

For sending emails from Python using your Gmail account IMHO best is to use yagmail packet.

You can waste time with smtplib library, but if you are using a Gmail account, just use yagmail.

Install it with:
pipenv install yagmail

Code examples

This is just a simple code example, for production code do not put your password in the source code.

Personally, I use one JSON file (this file is committed to source control with dummy credentials) per project for storing usernames and passwords for my personal projects.

This is a simple one line example:

I usually use it like this:

Adding image in the body of the email, you need to use yagmail.inline:

My advice and experience

Gmail have daily limits of how much emails can you send.

At the time of this writing, it is 500 emails per day, if you exceed it you will not be able to send emails for next 24h (at least that happened to me, and I have sent only 100 emails in a row).

So, my advice is to have separate Gmail account for just for sending emails automatically from your code, you do not want to lose the ability to send email from your personal email account (I was there and it is not funny).

Also, have some delay between sending emails, when I got my account locked for 24h I have only sent around 100 emails in a row, but after I have added time.sleep(15) I never had that problem again.

I am not using Gmail to spam people, I am using it for sending the email report to myself, mostly from web scraping tasks (around 20 per day).

If you have error 534

Solution is

Google blocks sign-in attempts from apps which do not use modern security standards (mentioned on their support page). You can, however, turn on/off this safety feature by going to the link below:

Go to this link and select Turn On
https://www.google.com/settings/security/lesssecureapps

From:
https://stackoverflow.com/a/26852782/2006674

Conclusion

Sending emails from Python is easy with yagmail.

Understanding Monty Hall dilemma with hacker statistics

Published on: 01.09.2018

With help of hacker statistics, the proof will be provided that in Monty Hall game you should always switch doors because you are doubling (from 33% to 66%) your success rate.

What is hacker statistics

In hacker statistics, you run simulations to calculate the probability of some event.

For example, the probability of the coin toss game is 50% heads, 50% tails.

Let say, for the sake of argument that you do not believe it.

Believe me, there are people who do not believe.

You could calculate the probability and after calculating you would get 50% for heads and 50% for the tails.

But you still do not believe or you think that maybe your calculation is wrong.

Then you can test it in practice.

Take a coin flip it a 100 times(or even a million) divide number of heads with the number of coin flips and the probability of heads is calculated.

As you can see, this is very time-consuming, especially if you want to run the experiment again.

An alternative to the manual coin toss is to write a computer program that will run the simulations and calculate the probabilities.

This is a much wiser approach because you can run it multiple times and it is faster (an average computer can do million of coin toss simulation per second).

Monty Hall game

I heard of Monty Hall game a few years ago in movie 21.

Here is the scene from the movie 21 with Monty Hall described.

Basic idea is that you have 3 doors, behind one door is a car, and behind other 2 doors is a goat.

You can choose only one door of three.

If you chose a door that has a car you get a car if you choose a door with a goat you get nothing (not even a goat).

The goal of the game is to choose a door with a car, but you do not know behind which door is the car.

So you choose some door, and probability for your success is 33%.

Monty Hall dilemma

After you have chosen one door, the game host will open one of the other two doors and goat will be shown.

The dilemma is the following: Is it in your interest to switch door?

The important thing to understand is that the game host will always open one door where is a goat.

Every single time, this will be done even when you choose a door with the prize or if you do not choose.

If the game host was only opening another door when you chose the door with the prize, then the best solution is not to switch the door.

If the game host was only opening another door when you chose the door with the goat, then the best solution is to switch the door.

Point is that every time game host will open another door where is a goat.

The answer to the puzzle is that if you switch door you will win 66% of the time.

This was counterintuitive to my understanding.

The best explanation, that I was able to understand I have found at https://www.youtube.com/watch?v=4Lb-6rxZxx0.

Monty Hall solution with hacker statistics

Altho I understood the math and logic why switching doors provide 66% probability of the win, I still did not believe that it is correct.

So I decided to make the computer program to simulate the game and solve this conundrum once and for all.

I did two simulations with a million iterations each.

The first simulation calculated the probability of win if you did not switch door.

No surprises there, the probability was 33%.

The second simulation calculated the probability of win if you did switch door.

To my amazement at that time, simulation shoved that probability of win if you switch door is really 66%.

Dou to my disbelief, I run it few dozens time, but each time probability of a win was at 66%.

I understood the math behind it, I did the simulation that proved math, but still, I had a hard time accepting that it is correct to switch door because it was counterintuitive to my own(false) reason.

Interesting to mention is that I needed few weeks to accept results of my own simulation.

Some interesting facts on Monty Hall

Wikipedia entry on Monty Hall is extensive.

A lot of interesting facts can be found.

Even experts have a hard time understand it:
“After the problem appeared in Parade, approximately 10,000 readers, including nearly 1,000 with PhDs, wrote to the magazine, most of them claiming vos Savant was wrong (Tierney 1991). Even when given explanations, simulations, and formal mathematical proofs, many people still do not accept that switching is the best strategy (vos Savant 1991a). Paul ErdÅ‘s, one of the most prolific mathematicians in history, remained unconvinced until he was shown a computer simulation demonstrating the predicted result (Vazsonyi 1999).”

One striking sentence is
“Pigeons repeatedly exposed to the problem show that they rapidly learn always to switch, unlike humans”

Are pigeons smarter than humans?

Interactive Monty Hall Game

If you still do not believe that switching door is the best strategy you can play it online.

Just do not switch tab, you will need to wait a few minutes.

Conclusion

In Monty Hall game you should always switch doors because you are doubling (from 33% to 66%) your success rate.

Learning Python design patterns Second Edition, book review

learning_python_design_patterns_se-Book_Cover

Published on: 15.08.2018

Number of pages: 141
Written by: Chetan Giridhar
Publish by: PACKT Publishing

Conclusion
This is the first book that I have read regarding software design patterns in Python language, for me it is useful as the code implementation reference.

Review

Book explains and implements some(singleton, factory, facade, proxy, observer, command, template method, MVC, state design) software design patterns in Python language.

It also gives the general overview of design patterns and antipatterns.

What I do not like is that book first does an explanation of code and then shows the code that has been already explained.

For me, it is better first to see the code and then to read the explanation of it.

The best approach for explaining code so far I have seen in Fluent Python.

In that book, the first code is shown and then it is explained.

And the best part, in Fluent Python, not the whole code is explained but just concepts that you have not seen in the book before.

My experience with learning software design patterns from books is generally unsuccessful.

I learn things when I need to use them, so generally only when I had the unmaintainable code and when I wanted to make it more manageable I usually end up reinventing some pattern.

Then I was able to understand some design patterns, from standpoint of why to use it (what is the benefit).

But I do recommend reading books about design patterns, even if you do not understand it completely because probably you will remember it when you need it in practice and then you can open it, read it and implement it.

Believe me, it is easier than reinventing patterns from scratch.

I think that book would be better is first there was code written without design patterns because than the reader could see what kind of mess it is and how hard is it to maintain it.

But that kind of book would probably be even 3-5 time longer.

How to backup personal GitHub repositories

Published on: 01.08.2018

I will show how to do a backup of your GitHub repositories with python-github-backup

Why to bother with a backup of GitHub

I can already see that there will be comments regarding why to do the backup of GitHub.

  • “It is a waste of time.”
  • “GitHub internally already have backups.” (I hope so)
  • “They will not lose your code” (But maybe I will)
  • “They will not go overnight out of business.”

Response to all those comments is:
You will not be worst off if you have your own backup.

If forever reason (GitHub go under, all repositories deleted by accident, alien attack) GitHub is not available anymore, I have my own backup of code that I have written.

Paid solution

If you are looking for a paid solution, BackHub looks like a good solution.
I have no experience with BackHub, nor am I in any way associated with it.

Free solution

After researching all available options I have decided to go with python-github-backup because it had more stars and contributors on GitHub than other projects.

I have used the number of stars and contributors on GitHub as the assumption that python-github-backup is more in use than other solutions so there are more people who will continue to support it in future.

In order to access your GitHub personal data, you need to have a personal access token.

After that, you can install it with pip/pipenv:
(I have installed it in separate virtualenv)

pipenv install github-backup
or
pip install github-backup

Run it with:
/full/path/github_backup/venv/bin/github-backup sasa-buklijas -t your_personal_access_token -o /full/path/github_backup --all

This command will backup all your GitHub information to /full/path/github_backup directory.

It would be tiresome to run this command every day, so I have automated it on my online hosting.

Crontab

My usecase:
15 00 * * * /full/path/github_backup/venv/bin/github-backup sasa-buklijas -t your_personal_access_token -o /full/path/github_backup --all > /full/path/github_backup/last_log.txt

> /full/path/github_backup/last_log.txt is used to have an output of the last backup command.
>> can be used to have outputs of all backup commands, but I have found that having just last one is enough.

Conclusion

“You will not be worse off if you have your own backup.”

Do you have backups of your own personal GitHub repositories, if you have, what do you use for backup?

Subnetting, book review

subnetting-Book_Cover

Published on: 15.07.2018

Number of pages: 52
Written by: Todd Lammle
Publish by: Sybex

Conclusion
This book will teach you how to do subnetting of the IPv4 network.

Review
It is easy to read because the book is only 52 pages, what is more than enough to explain subnetting of the IPv4 network.

Has a lot of examples so it is easy to follow.

The first chapter is subnetting basics, and then there are 3 chapters for A, B and C class.

Beginning serverless computing, book review

beginning_serverless_computing-Book_Cover

Published on: 01.07.2018

Number of pages: 199
Written by: Maddie Stigler
Publish by: Apress

Conclusion
Interesting read, it shows the current state of AWS, Azure, and Google Cloud.

Review

Chapter 1 is explaining what is serverless computing.

The second chapter gives an overview of the current market.

Next three chapter give example how to implement it in AWS, Azure, and Google Cloud.

The last chapter is demonstrating the agnostic approach.

Serverless computing is the topic that interests me a lot.

I see more advantages than disadvantages with serverless setup, especially cost (if I am charged by used resources and not by subscription) and scaling.

Exercises for Programmers, book review

exercises-for-programmers-Book_Cover

Published on: 15.06.2018

Number of pages: 118
Written by: Brian P. Hogan
Publish by: The Pragmatic Bookshelf

Conclusion
Book has good (real life) exercises that can be applied to learn a programing language.

Review
This book will not teach you programming but have 57 exercises that you can do in any language.

Each exercise has few additional challenges, so there is enough material for practice.

The only way to learn new programming langue(or your fist) is to make something concrete and this book can provide that.

Flowcharts are also shown in the book, I think that is cool.
Today flowcharts are mostly forgotten but are useful to know.

If you are not experienced programme (I do have 15+ years of development) my suggestion is to first solve an exercise by any means necessary and after you can do it with constraints and lastly also challenges from the book.

If some exercise is too hard (you have spent few hours and still no progress) just go to next one.
You will have more motivation to learn if it is fun and not dreadful.
After you finish some other exercises and get more experience, you can come back to old ones.

The best part about exercises in this book is that they are pretty close to real life programming, so you are not wasting your time (like on Codility) solving some problems that you will never encounter in real life software development.

My solutions can be found at https://github.com/sasa-buklijas/57-challenges-to-develop-your-coding-skills/.