SED command


sed ‘s/find/replace/’ filename

Find and replace a word in a file

Ex output:

sed ‘s/find/replace/1’ filename

It will find and replace in line 1

sed ‘1,3 s/find/replace/1’ filename

It will find and replace with the given range

Ex output:

sed ‘s/find/replace/g’ filename

Find and replace all the occurrences in a line.

sed ‘1,5 s/setof/set/g’ auto.tcsh |sed ‘10,15 s/e/0000/g’  |head -15

Replace the string within the given range.

Unix commands Session


In the layout team, we are learning UNIX command with the support of Sanjeev to improved our skills for automating tasks.

Session on process, kill, disk utility, alias, pushd, popd, tar, and gzip commands:

Process command:

To find process which are all running use “ps” command

ps -ef ; print each (all) process

ps -u <userName>; print process by user

ps -g <groupName>; print process process bu group

groups <username> ; to find group in which user is.

Kill Command:

Used to kill process

kill <signal> <pid> ; send a signal to process <pid>.

kill -l ; signal list;

kill -9 <pid>; -9 is a kill signal

kill -9 -1 ; all process it can.

xkill ; Select the window whose client you wish to kill with button 1….

Disk utility commands:

du -b <path> ; print file space in bytes

du -m <path> ; print file space in megabytes

du -k <path> ; print file space in kilobytes

du -h <path> ; print file space in human readable form.

du -d <depth> -h ; print all files space in given <path> with a given depth.

df -h ; print mounted disk space detail in human readable form

  • To find biggest file:
    • du -b | sort -nr ; sort file based size (bytes), In sort è n = combine digits into number, r = reverse order
  • To find smallest file:
    • du -b | sort -n ; sort file based size (bytes), In sort è n = combine digits into number
Send process into background and bring it foreground:

Press Ctrl+z to suspend process.

Type bg to send process into background

Type “jobs” to get job ID.

Type “fg %<job ID>” to bring process into foreground

Alias: set a alternate name to a command or a group of command within quotes separated “;”

alias ls ls -la –color=auto ; set “ls” as a “ls -la –color=auto”.

alias DoYouWantContinue ‘echo “\nDo you want to continue (Y/N):” ; set input = $< ; if($input != “Y” && $input != “y”) exit’ è set DoYouWantContinue as a set of command separated by “;”.

Unalias <alias Name> ; to remove alias;

/<command> ; to run raw command; even when command name is aliased to something else.

Change directory :

cd <path> ; change directory to given path.

cd – ; last cd path.

pushd <path> ; push directory into stack and cd to it.

pushd +<rotate stack number> ; rotate the stack

popd ; pop the top directory in stack

Create Archieve:

tar -cf <archive.tar> <directory> ; Create archive.tar from <directory>, it is like git version control.

tar -tvf <archive.tar> ; List all files in archive.tar verbosely.

tar -xf <archive.tar> ; Extract all files from archive.tar.

tar -d -f <archieve.tar> <directory>; show the difference between tar and files in directory.

tar -u -f <archieve.tar> <directory>; create a new entry for all files with new time stamp(if file is modified).

tar -r -f <archieve.tar> <directory>; append a new file in tar file.

tar -xvf auto.tar test.tcsh –occurrence=2 ; extract 2nd occurrence of test.tcsh file in auto.tar tree.


gzip -c <file> ; compress file

gzip -d <file> ; decompress file

gunzip <file> ; unzip file


No Code / Less Code Machine Learning/Deep Learning

~ Vimal 

As a novice, one will feel alienated with the buzz words and vocab used in the context of ML/DL world. To bring more people into the AI scenario, several of the tools are being developed by researchers and companies worldwide. They frame this with the phrase “Democratizing AI”. The corporates of course focus mainly on the subscription business model for enabling one to make their custom machine learning / deep learning model. Whereas few of the academia geeks focus on developing tools to bring the common man into the AI world. Some of the examples include

  • Ktrain
  • Ludwig
  • Fastai

In this blog post I wish to introduce a simple and easy to use library ktrain. This is a high level wrapper library for the Tensorflow keras(a low level API like library for the TF). Using this library we can  build, train & deploy a deep learning model at ease. It also provides interfaces and includes functionalities from other SOA deep learning library PyTorch(from FB Research).It abstracts the various processes one will encounter while building a deep learning model for his/her use case. It provides support to create models for text & image processing workflows. Text processing or widely referred as Natural Language processing is the toughest job every machine learning engineer will encounter. There will be a lot of bias in selecting the different pretrained models for different tasks(sentiment analysis, text classification, text generation, Q/A systems etc.). This ktrain library comforts the user by doing all the heavy lifting. A text classification pipeline will only require 3 lines of python code to train and build a model. It also possesses the methods to save the model for predicting the unknown datasets. One catch is the zeroshot classifier pipeline which supports the user to classify the text/sentences/documents without training. It has support to import all the transformer models like BERT, distilBERT, XLNet etc. Image classification task is easily achievable with the pretrained model like ResNet50, Inception etc. It supports graph & tabular dataset also to enable the user for custom building a model.

A quick tutorial to execute a zeroshot sentiment analysis classification(binary classification – Positive/Negative)

  1. Install the ktrain through pip 
  2. Execute the code

With the introduction of transfer learning capacities(through transformer architecture) to the natural language related tasks, the NLI/NLU/NLG(Natural Language Inference/Understanding/Generation) field is speeding up in the last couple of years paving the way to surpass the human capabilities in all the language related tasks.

Python course

~Logeshwari, Sandhiya

We had interns coming to STEMland to learn to program. A few of them were learning Scratch and making projects. We conducted a course on the basics of Python. It was a five-day course for one and a half hours each day. There were about 15 of them attending the course. Everyday based on what the interns learned assessments were given accordingly.

People shared their assessments with others at the beginning of the class for about 10 minutes.

One participant sharing his assignment

In the five days of the course, everyone learned the following.

  • Print statements
  • Variables
  • Data structures/data types (tuples, list, dictionaries, etc.)
  • Data type conversions
  • Functions
  • Imports
  • Global variables
  • Scope of functions
  • Classes

This is the first time that I took a technical course. It was a new experience of teaching/ sharing what I learned to adults.  I planned for the course and I decided the assignments before the day which made me clear on what I wanted to do. Instead of going to more advanced it is always good to review the basics not only in programming. It was a nice opportunity for me to review my basics in python

Where to improve:

I felt that I need to follow the time properly. I missed a few important things which I added later which I should have told initially itself.

Graph with Abinash

Abinash a student from the 8th grade at Udavi school started his work on graph and initially had a little difficulty plotting points. He wanted to create a game for others to understand the concept and so together we started to mind storm on how and what the game should do.

Then he saw a picture on his mathematics text book and decided to create a point follower game at the end of which the user will get an image.

The program was coded in Scratch.



mysql client documentation-windows


What is mysqlclient?

The client package also comes with utilities that allow you to easily backup/restore data and administer the server.

Install mysqlclient for windows

Go to the link:

Download the file which version supported your system

Download the file: mysqlclient-1.4.4-cp37-cp37m-win32.whl

open the terminal: type pip install mysqlclient-1.4.4-cp37-cp37m-win32.whl


Documentation on debugging issue for creating database

While running the crawler code to extract data about the categories(Beauty -> Tools and Accessories  products) in,  I had come across the following error. I will be showing what I had done to rectify the error.

runfile(‘C:/Users/Dell/Documents/Quilt/quilt-beauty/quilt-beauty/’, wdir=’C:/Users/Dell/Documents/Quilt/quilt-beauty/quilt-beauty’)

Error tokenizing data. C error: Expected 7 fields in line 68, saw 8

If you look at the database, the database will be empty.

To look at the database, open the ‘db.sqlite3’ file using DB Browser (SQLite) application.

To rectify the error, we need to open the .csv in a normal word editor e.g. notepad++

You will be able to see the ‘ ” ’ double quotes at the beginning and at the end.

Remove the double quotes and save the text file.

NOTE: If there are already data in the database, to remove the data from the database do the following.

Open the db.sqlite3 file using DB Browser (SQLite) application.

Go to ‘Execute SQL’ tab

Type the following command

delete from file_name

delete from beauty_skin_categories

The above command will delete the data that was present in the database.

To execute the command run the command, click the run button.

After running the command click ‘Write Changes’

Then open Spyder and run the code. This time it should work.

The database for the beauty_skin_categories table will be created.


How to use Git in Eclipse(Cloning a respiratory)

–  Prabaharan

1.Open your Gitlab

To copy the URL that you want to clone in your local respiratory.

Go to: Project -> Details

Select “Clone” and under “Clone with HTTPS”press on the “copy URL to clipboard”


2. Open Eclipse

Go to:  Windows -> Preference -> Open Preferences -> Other

‘Open Properties’ window will pop up

Select ‘Git folder’ in that as shown below and press ‘Open’

‘Source Git Repository’ window will pop up

Paste the copied URL in the URL text box under the ‘Location’ tab

Also provide the ‘User ‘ and ‘Password’ for under the ‘Authentication ‘ tab.

Press Next

‘Branch Selection’ window will pop up

Select the respective branch that you need

Press Next

The directory where the repository to be cloned will be shown.

After that select ‘Finish’

Select the folder which you want to import.

Right click on the folder and select ‘Import Projects’ from the dropdown.

This will lead to location of your local directory.

Click ‘Finish’ to complete.




Sample WebScraping

########### Source code for webscraping ############

from selenium import webdriver
from bs4 import BeautifulSoup
import pandas as pd
driver = webdriver.Chrome("/home/ranjith/Documents/Software/Chrome/chromedriver_linux64/chromedriver")
products=[] #List to store name of the product
prices=[] #List to store price of the product
content = driver.page_source
soup = BeautifulSoup(content)
for a in soup.findAll('a', attrs={'class':'_31qSD5'}):
    name=a.find('div', attrs={'class':'_3wU53n'})
    price=a.find('div', attrs={'class':'_1vC4OE _2rQ-NK'})
    rating=a.find('div', attrs={'class':'hGSR34 _2beYZw'})
df = pd.DataFrame({'Product Name':products,'Price':prices}) 

######## OUTPUT ###########

                                        Product Name    Price
0  Apple MacBook Air Core i5 5th Gen - (8 GB/128 ...  ₹67,990
1  HP 15 Core i3 6th Gen - (4 GB/1 TB HDD/Windows...  ₹28,997
2  Apple MacBook Air Core i5 5th Gen - (8 GB/256 ...  ₹92,500
3  Lenovo Core i5 7th Gen - (8 GB/1 TB HDD/DOS/2 ...  ₹52,990
4  Lenovo Core i3 6th Gen - (4 GB/1 TB HDD/Window...  ₹30,550
5  Lenovo Ideapad Core i3 6th Gen - (4 GB/1 TB HD...  ₹32,990
6  Lenovo Core i5 7th Gen - (8 GB/2 TB HDD/Window...  ₹75,500

GCD and LCM using Sympy in Python

##########Source code for finding GCD and LCM using Sympy in python#########
from sympy import *
x = Symbol('x')
expression1, expression2 = x**2 + 4*x - 12, x**2 - 5*x + 6
def findGCD(expression1, expression2):
    print("########### Find GCD ############")
    factorsExpr1, factorsExpr2 = factor(expression1), factor(expression2)
    print("factorsExpr1 : ", factorsExpr1)
    print("factorsExpr2 : ", factorsExpr2)
    GCD = gcd(expression1, expression2)
    print("GCD : ", GCD)
    return GCD
def findLCM(expression1, expression2, GCD):
    print("########### Find LCM ############")
    factorsExpr1, factorsExpr2 = factor(expression1), factor(expression2)
    LCM = factor(lcm(expression1, expression2))
    print("LCM of Expr1 and Expr2 : ", LCM)
    print("f(x)*g(x) = ", factor(expression1*expression2))
    print("(f(x)*g(x))/GCD = {0}/{1}".format(factor(expression1*expression2),GCD))
    LCM = simplify((factorsExpr1*factorsExpr2)/GCD)
    print('Finding LCM using (f(x)*g(x))/GCD" : ', LCM)
    return LCM
GCD = findGCD(expression1, expression2)
LCM = findLCM(expression1, expression2, GCD))
######## OUTPUT #############
########### Find GCD ############
factorsExpr1 :  (x - 2)*(x + 6)
factorsExpr2 :  (x - 3)*(x - 2)
GCD :  x - 2
########### Find LCM ############
LCM of Expr1 and Expr2 :  (x - 3)*(x - 2)*(x + 6)
f(x)*g(x) =  (x - 3)*(x - 2)**2*(x + 6)
(f(x)*g(x))/GCD = (x - 3)*(x - 2)**2*(x + 6)/x - 2
Finding LCM using (f(x)*g(x))/GCD" :  (x - 3)*(x - 2)*(x + 6)