How to run python script on multiple cores/threads (multiprocessing)

I’ve been looking for means of parallel execution in Python for quite a while. There’s a bunch of various libraries like Dask and Joblib Parallel. However using multiple threads won’t give you better performance due to the global interpreter lock called GIL. At the moment the best solution I’ve found is multiprocessing. It uses subprocesses … Read more

Calculate execution time in Python

In any Python script you can easily count execution time. In Jupiter notebooks simply use built-in magic commands like %%time in the very beginning of the cell. Now with the IDEs like PyCharm you should use something like this: import datetime as dt t1 = # your code here t2 = ft = … Read more

Fatal error C1083: Cannot open include file: ‘io.h’: No such file or directory

Cannot open include file 'io.h' No such file or directory error command failed with exit code 2

I’ve stumbled across this annoying error when trying to setup Cython working environment. Here are a couple of good guides: one, two. Everything went fine until I tried to compile the Cython code with python build_ext –inplace Then I got this nasty error: C:\Users\irmsc\anaconda3\envs\backtesting\include\pyconfig.h(59): fatal error C1083: Cannot open include file: ‘io.h’: No such … Read more

Useful Anaconda terminal commands

Create Anaconda environment: conda create –name MyEnv Activate virtual environment on Windows: activate MyEnv Install package conda install package-name Activate virtual environment on LINUX, macOS: source activate MyEnv Clone base Anaconda environment: conda create –name MyEnv –clone base Deactivate current environment: conda deactivate List all packages and versions installed in active environment: conda list Get … Read more

What the heck is ‘www-data’

The ‘www-data’ is the default user of Nginx and Apache servers on Ubuntu system. That’s why when we need to serve some files or CMS with Nginx we change the ownership of the folder to: www-data:www-data If you are wondering why there’s 2 ‘www-data’ separated by colon, it’s because first is the username and second … Read more

Useful and important commands for mySQL & MariaDB

Create database: CREATE DATABASE database1; Create user CREATE USER user IDENTIFIED BY ‘secure_password’; Grant all privileges on specific database: GRANT ALL PRIVILEGES ON database1.* TO user; Show all users: SELECT User FROM mysql.user; Show all databases: SHOW DATABASES; Delete user: DROP User ‘user’@’localhost’; Remove database: DROP DATABASE YourDatabase; Reload the GRANT tables to enable the … Read more

Select dataframe rows by specific column string values in Pandas

Hello, I’m back! 😎 Now the traditional method is this: df.loc[df[‘column_name’] == value] For string, and for numeric values this: df.loc[df[‘column_name’] == ‘string’] While it always work with numeric values, for string values sometimes it doesn’t work. It picks up a blank dataframe. I guess it’s something to do with encoding of the source where … Read more

Best Linux distribution for MySql database

I’ve been researching the web for this topic for a couple of hours and came to a conclusion that savvy programmers prefer CentOS for running their MySql and other databases. Here’s an interesting poll. Yes, it’s dated but still, all the fresh info I found on this topic tend to stick with the same choice. … Read more

Command to show information about my linux distribution

The quickest way to find out is to type: cat /etc/*-release in my case the result is: CentOS Linux release 8.3.2011 NAME=”CentOS Linux” VERSION=”8″ ID=”centos” ID_LIKE=”rhel fedora” VERSION_ID=”8″ PLATFORM_ID=”platform:el8″ PRETTY_NAME=”CentOS Linux 8″ ANSI_COLOR=”0;31″ CPE_NAME=”cpe:/o:centos:centos:8″ HOME_URL=”” BUG_REPORT_URL=”” CENTOS_MANTISBT_PROJECT=”CentOS-8″ CENTOS_MANTISBT_PROJECT_VERSION=”8″ CentOS Linux release 8.3.2011 CentOS Linux release 8.3.2011 If you are interested what command cat /etc/*-release means … Read more