Tag Archives: python

Installing Python OpenCV and other libraries

Here, we will be installing the libraries that are required to perform image processing operations. We will be using the following Python libraries

OpenCV
Numpy
Matplotlib

Why we are using OpenCV, not Matlab or any other?

Because it is open source, fast(written in C/C++), memory efficient and easy to install (can run on any device that can run C).
if you really want to learn about how computer vision works from the initial steps to the last, I suggest you learn OpenCV first then use any deep learning library like Tensorflow, pytorch etc when you’re ready to train a deep learning algorithm for better performance/accuracy results.

Installing Numpy, OpenCV or any library (For WINDOWS)

There are two ways to install any library in Python IDLE,

using pip command: In the installed Python folder, go to Scripts folder and open command prompt(press and hold Shift + Right Click and select Open command window here). Then write pip install library name to get it installed. For example

Python

pip install numpy

1

pip install numpy
using .whl file: First download .whl file of any library (version corresponding to your Python) from here. Then open the command prompt where you have downloaded this file and write pip install filename.whl. For example

Python

pip install numpy‑1.14.2+mkl‑cp36‑cp36m‑win32.whl

1

pip install numpy‑1.14.2+mkl‑cp36‑cp36m‑win32.whl

Installing Numpy, OpenCV or any library (For UBUNTU)

First, install pip using apt-get, then you can install any library as shown below

sudo apt-get install python-pip  
sudo pip install numpy

1 2	sudo apt-get install python-pip sudo pip install numpy

Anaconda: open the Anaconda prompt and write pip install numpy or any other library name which you want to install.

Now, you might have got some feeling about how to install Python libraries. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Python If name == “main”

1 Reply

There are two things which we can do with our script/program, either we run it directly or import it into another program. Suppose we have a my_module.py file, as shown below

# my_module.py file

def function():
    print ('Hello World')

function()

# my_module.py file

def function():

print ('Hello World')

function()

To run the my_module.py file directly we write python my_module.py on the command line. But if we want to import this file, then we have to comment out the function() calling statement. Import file means we only want functions, classes etc. and no code execution statements like a function call from that file. Just let the user call these at his own will.

For example, import math only imports functions from the math.py file. It is up to us whether we call math.sin() or math.cos(). That’s why we should remove code execution statements before importing a file.

Problem: “Commenting out function call from the file which we want to import”. For large modules, this is seriously a headache.

Question: Can’t we do something to automate the above problem, meaning writing import should automatically comment out or remove the function call.

Solution: Python if __name__ == “__main__” statement.

Concept: Before executing the code, Python defines a few special variables. For example, if the Python interpreter is running that module (the source file) directly, it sets the special __name__ variable to have a value “__main__”. If this file is being imported from another module, __name__ will be set to the module’s or file’s name.

So, what we do is write all the code execution statements inside if __name__ == “__main__” statement. When we run the file directly this condition is satisfied and the code is executed. While importing __name__ == file_name and not __main__ so the code is not executed. Let’s see an example

# file hello.py
def func():
    print("func() in hello.py")

print("Hello World")

if __name__ == "__main__":
    print("run directly")
    func()
else:
    print("imported")

# file hello.py

def func():

print("func() in hello.py")

print("Hello World")

if __name__ == "__main__":

print("run directly")

func()

else:

print("imported")

Running the above hello.py file directly (F5 in shell or by command line), we get this output

Hello World
run directly
func() in hello.py

Hello World

run directly

func() in hello.py

Now, import this hello.py into another file say hi.py like as shown below

# hi.py file
import hello

1 2	# hi.py file import hello

Running the above code gives the output shown below

Hello World
imported

1 2	Hello World imported

Clearly, the func() is not executed while importing and this is what we want.

Now, you might have got some feeling about the Python if __name__ == “__main__” statement. Always use this. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Python Variables and Scope

1 Reply

To understand what is happening in the code, you should know what the “scope” of variable means. The scope of a variable refers to the places from where we can see or access a variable. To understand this more clearly, we should first know what “variable” actually means.

A variable is a reference to a memory address that contains the value. Let’s see the figure below.

Suppose, we have x=5. There are two ways of reading this line.

‘x’ is a variable name that references to a memory location in which 5 is stored.
variable x equals 5

Saying the humungous first line every time may be frustrating, so we usually use the second way. Here comes the confusion. Let’s see by an example

x = 5
def number():
    x = 3
def numb():
    x = 6
number()
numb()    
print x

x = 5

def number():

x = 3

def numb():

x = 6

number()

numb()

print x

Question: How many variables are there in the above code?

If we use the definition x equals 5, we might think that it is the same variable x with different values. But if we use the actual definition of the variable, we can argue that there are 3 variables as they refer to different memory locations and thus are different. We can always check the identity of the location of the object in memory using the python id() function. Just write id(variable_name) which outputs a number that refers to some memory location(same number means the same variable). So, use the shortcut definition but always remember the actual meaning.

This means we can define a number of variables with the same name(say x) in a program without having a relation between them. This sometimes creates confusion. Let’s understand this by an example

x = 5
def number():
    x = 3
number()    
print x

x = 5

def number():

x = 3

number()

print x

Now, which value should this x prints? This is where the scope comes into the picture. Scope tells us which variable we can access and from where.

LEGB Rule: LEGB stands for Local, Enclosing, Global and Built-in. This tells us the order in which python accesses the variable in a program. First, python will check if the variable is present in the local scope, if not then checks in enclosing scope and so on till the builtin scope is checked.

Let’s understand the LEGB terms one by one:

Local: Python first tries to search for an identifier(variable) in Local scope. The local variable exists only within the block/function that it is declared in. When that block/function ends, the local variable has no significance, it gets destroyed. We cannot use it outside the function where it is declared. For example in the code below, x=3 is local to the number() function means we cannot access that variable outside this function. So, the output of the last print(x) will be an error saying name ‘x’ is not defined.

Python

def number(): x=3 print x number() # 3 print x # name 'x' is not defined

1
2
3
4
5

def number():
    x=3
    print x
number()         # 3
print x          # name 'x' is not defined

Enclosing: Enclosing refers to the nested functions where one encloses the other. If Python does not find an identifier(variable) within the local scope, it will examine the Enclosing scope to see if it can find the variable there. Let’s understand this by an example

def outer():
    x=4
    def inner():
        print x           #----- 1
    inner()
    print x               #----- 2

outer()

def outer():

x=4

def inner():

print x #----- 1

inner()

print x #----- 2

outer()

For the print x —1, inner() is Local and outer is Enclosing while for the print x—2, outer() is Local. So, print x—1 will first search for x in the inner(). Since there is no x in the inner() so we go to outer()/Enclosing and check if x is there or not. So, print x—1 outputs 4. For print x–2, we have x in the Local scope of outer() so this will output 4.

Global: Global variable means that it is accessible from anywhere in your script, including from within a function. It is usually defined at the top of the script or outside of the function. If Python fails to find the variable in both above scopes, then it will check the global scope. Let’s see an example

x=4
def outer():
    def inner():
        print x           #----- 1
    inner()
    print x               #----- 2

outer()

x=4

def outer():

def inner():

print x #----- 1

inner()

print x #----- 2

outer()

As we have no x in the local and enclosing scope, so Python checks the global scope and outputs 4 for both print x statements.
Note: using too many global variables will generally lead to name conflicts and will likely lead to unwanted side effects.

Built-in: Built-in means the function and variables that are already defined in python. If an identifier is not found in any of the above scopes within a module, then Python will examine the built-in identifiers to see if it is defined there. For example

Python

x = len ( 'PANKAJ' ) print x # 6

1
2

x = len ( 'PANKAJ' )
print x                          # 6

If len is not found in any of the scopes, then Python would consult the Built-In scope, where it will find the len function and outputs 6. But if we create our own len function then it will have precedence over Built-in len function. Let’s see by an example

Python

def len(x): return (1) x = len ( 'PANKAJ' ) print x # 1

1
2
3
4

def len(x):
    return (1)
x = len ( 'PANKAJ' )
print x                          # 1

Since our len function precedes over built-in len and thus the output of print x statement equals 1.

Now, I hope that you might have understood the Python variable and scope, Let’s look at some of the keywords that are useful regarding this

Global Keyword: With this, we tell the Python to use the globally defined variable instead of locally defining it. This is useful when we want to change the value of a variable through a function. To use it, simply type global followed by the variable name. Let’s see an example

Python

x=4 def change(y): global x x = y change(10) print (x) # 10

1
2
3
4
5
6
7

x=4
def change(y):
    global x
    x = y

change(10)
print (x)                  # 10

NonLocal Keyword: This is useful in nested functions. It causes the variable to refer to the previously bound variable in the closest enclosing scope. Let’s see by an example

def outer():
    x=4
    def inner():
        nonlocal x
        x = 10
        print (x)
    inner()
    print (x)

outer()             # output 10,10

def outer():

x=4

def inner():

nonlocal x

x = 10

print (x)

inner()

print (x)

outer() # output 10,10

By nonlocal, we tell Python that this x is same as defined in the enclosing outer() function.

Now, you might have got some feeling about the Python variable and scope. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Side Effects in programming

Leave a reply

According to Wikipedia, a function or expression is said to have a side effect if it modifies some state outside its local environment. Let’s understand this in depth by examples.

To understand side effects, first, we should know what “state” is. In layman terms, the state refers to the value of the variable at any given point in the program like x=2.

Now, the above Wikipedia definition might have started to make some sense. If not, don’t worry, see the figure below.

In this figure, we have a variable x that is outside the scope of two functions. Clearly, after the Function_2 call, the value of x changes which means that Function_2, other than returning value(if any), is interacting with variables outside its local environment and thus has a side effect.

In general, any lasting effect that occurs in a function, not through its return value, is called a side effect. Not through its return value means if we assign x=Function_1(), this will clearly change the value of x but this is not a side effect. For side effect, the change should occur inside a function.

Common ways to have side effects:

Modifying a non-local variable: Let’s understand by an example

x=3
def square(x):
    x=x*x
    return x
def square_side_effect():
    global x
    x=x*x
    return x
print(x)                 # 3
print(square(x))         # 9
print(x)                 # 3 
print(square_side_effect())     # 9
print(x)                 # 9

x=3

def square(x):

x=x*x

return x

def square_side_effect():

global x

x=x*x

return x

print(x) # 3

print(square(x)) # 9

print(x) # 3

print(square_side_effect()) # 9

print(x) # 9

The x in the square() function is local to its scope so no change in outside x. Global in square_side_effect() makes x global so any change in x will change it in whole code (outside function scope) so the square_side_effect function() is not pure(has side effects).

Changing the value of a mutable object: Let’s take Lists which is a mutable object.

lst = [1,2,'Pankaj']
def list_side_effect(lst):
    lst[1] = 1
    lst[2] = 'Kang'
    return lst
print(lst)             # [1,2,'Pankaj']
list_side_effect(lst)
print(lst)             # [1,1,'Kang']

lst = [1,2,'Pankaj']

def list_side_effect(lst):

lst[1] = 1

lst[2] = 'Kang'

return lst

print(lst) # [1,2,'Pankaj']

list_side_effect(lst)

print(lst) # [1,1,'Kang']

Clearly, after calling list_side_effect(), the value of lst changes so, list_side has side effects.

Performing Input/Output operations: This doesn’t change any objects or variable value, but it does have a potential lasting effect outside the function call because a person might see the output and be influenced by it.
Calling other side-effect functions

Now, you might have got some feeling about the side effects. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Expressions vs Statements

Leave a reply

Statement and expression are two important terms which are commonly misunderstood. Let’s start the explanation from the expression term.

Expression:

An expression is a combination of values, variables, and operators including function call operator (), subscription operator [ ] etc. I guess that it is not clear, so let’s see an example.

>>> 1+1
2
>>> sum(range(2*3))
15

>>> 1+1

>>> sum(range(2*3))

In simple terms, an expression produces a value (1+1 is an expression as it produces a value 2) and can be written wherever a value is expected, for example as an argument in a function call, for example, 2*3 is expression given as an argument to range function in the code above.

Note: If you type an expression on the command line, the interpreter evaluates it and displays the result. While in a script an expression all by itself is a statement. You have to write print(expression) in order to display the result.

A value or variable is considered an expression as the interpreter returns the result(See example below). Functions that do not return anything is not an expression.

>>> 4
4
>>> x
2

>>> 4

>>> x

Note: Expressions can be reduced to any kind of value, which can be any integer, string or even a Python object like

>>> map(lambda x: x/2 , range(6))
<map object at 0x0000006242848588>

1 2	>>> map(lambda x: x/2 , range(6)) <map object at 0x0000006242848588>

Note: Evaluating an expression is not quite the same thing as printing a value. Expressions are displayed by the interpreter in the same format we enter, but print statement only shows the content (See example below)

>>> name= 'Pankaj'
>>> name            #  expression
'Pankaj'
>>> print(name)     #  statement
Pankaj

>>> name= 'Pankaj'

>>> name # expression

'Pankaj'

>>> print(name) # statement

Pankaj

Statements:

A statement is an instruction that the Python interpreter can execute. In short, every line of python code is a statement. Note that expressions are statements as well. Below is an example of print and assignment statements.

>>> x = 2        #  assignment (Do not display result)
>>> print(x)     #  print
2

>>> x = 2 # assignment (Do not display result)

>>> print(x) # print

Expression Statements: In this, first the expression is evaluated then the statement is executed. In the example below, the 2*3 expression is evaluated first, then the print statement is executed.

>>> print(2*3)
6

1 2	>>> print(2*3) 6

Now, you might have got some feeling about the difference between expression and statement. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Functional programming paradigm

Leave a reply

Functional programming is based on Lambda Calculus, a computational model proposed by Alonzo Church in the 1930’s. This paradigm falls under the declarative umbrella rather than imperative, meaning, programming is done with expressions or declarations instead of statements. (Relax… See below the properties and you will understand what this means)

Note: We can do programming in a functional style in languages that are not specifically designed for functional programming. C++, Java, Python etc. all added constructs to facilitate the functional style. So, programmers using “mostly imperative” languages may have utilized some of the functional concepts.

Pros: more concise, more predictable, and easier to test than imperative or object-oriented code only if you understand.

Main Concepts of Functional Programming:

Pure Functions: A pure function is a function which follows given properties like

For the same input, it always returns the same output.
Has no side effects(For details check the link).
Referential Transparency.

Although, these properties are somehow correlated. Like, a function can only return the same output for the same input, only if it has no side effects. Let’s see by examples

def square(x):
    return x*x

1 2	def square(x): return x*x

The square function always gives the same output for the same x .

def random_num():
    return np.random.randn()

1 2	def random_num(): return np.random.randn()

Even though we didn’t pass any arguments to the random_num function, it gives different output, meaning that random_num is not pure.

Now, let’s look at the side effect property.
Side effect is any state(value of the variable) change outside the called function scope. In simple terms, if a function changes the state/value of the variable that is outside function scope or local environment then the function/unit of code is said to have a side effect.

x=3
def square(x):
    x=x*x
    return x
def square_side():
    global x
    x=x*x
    return x
print(x)                 # 3
print(square(x))         # 9
print(x)                 # 3 
print(square_side())     # 9
print(x)                 # 9

x=3

def square(x):

x=x*x

return x

def square_side():

global x

x=x*x

return x

print(x) # 3

print(square(x)) # 9

print(x) # 3

print(square_side()) # 9

print(x) # 9

Clearly, the x value changes after square_side() function which means that this function has side effects. The x in the square() function is local to its scope so no change in outside x. Global in square_side() makes x global so any change in x will change it in whole code (outside function scope) so the square_side function is not pure(has side effects).

Another important property of pure functions is Referential Transparency means we can replace a function call with its resulting value without changing the meaning of the program. Let’s see by an example

def double(x):
    return x*2
y = double(2) # or y = 4

def double(x):

return x*2

y = double(2) # or y = 4

In the above example, we can easily replace double(2) by 4 because we know that this function always gives the same result for the same input(No side effects). This is Referential Transparency.

First Class functions: “first-class” is a computer science term that describes something that has no restriction on its use. Thus, first-class functions mean it can appear anywhere in the program like as arguments to other functions and as their return values.
Recursion: Recursion means letting an operation be repeated until it reaches the base case. In the functional paradigm, we do not use the loops and all the work is done by recursion.

Python

def factorial(n): if n == 1: return n else: return n*factorial(n-1)

1
2
3
4
5

def factorial(n):
   if n == 1:
       return n
   else:
       return n*factorial(n-1)
Avoid Shared State: A shared state is a state that is shared between more than one function or more than one data structure and you should avoid creating functions that rely on the same variable.

Now, you might have got some feeling about the Functional programming. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Procedural Programming paradigm

Leave a reply

Procedural programming, as the name suggests, is based on the concept of procedural call. To understand it more deeply, we should first know what procedural call means.

Procedures, also known as routines, subroutines, or functions, simply contain a series of computational steps to be carried out. Let’s see an example of the procedure

#Procedure
def add_list(my_list):
    Total = 0
    for x in my_list:
       Total += x  
    return Total        
# Procedure Call
add_list([1,2,3,4])

#Procedure

def add_list(my_list):

Total = 0

for x in my_list:

Total += x

return Total

# Procedure Call

add_list([1,2,3,4])

Procedure calls are modular and are bound by scope. It means we break the program into independent, interchangeable small pieces known as modules such that each module contains everything necessary to execute. Each module is composed of one or more procedures.

Procedural programs may possibly have multiple levels or scopes, with subprograms defined inside other subprograms. Scoping helps to keep the procedures modular and prevents/allows the procedure, from accessing the variables of other procedures.

Procedural programming falls under the imperative umbrella which means that procedural inherits the properties of the imperative paradigm. Procedural programming uses a linear or top-down approach.

Pros: easier to read and more maintainable, more flexible, allows modules to be used again in the form of code libraries.

Cons: this approach may not use hardware efficiently when tackling complex problems because of side effects (may not run parallelly)

In short, procedural programming means breaking the program into modules and using the imperative approach. If we take away the imperative nature from procedural, then it becomes same as Functional.

Now, you might have got some feeling about the Procedural programming. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Imperative vs Declarative programming paradigm

Leave a reply

In this post, we will be looking at the major differences between imperative and declarative programming paradigms.

Imperative Programming paradigm: The languages that fall into this category have 2 main properties:

Control flow is explicit meaning how the operation took place, step by step or more naively “First do this then do that”.Thus, the order of steps is crucial.
Allow side effects.

To understand side effects, first, we should know what “state” is. In layman terms, the state refers to the value of the variable at any given point in the program like x=2.

Side effect is any state change outside the called function scope. In simple terms, if a function changes the state that is outside function scope or local environment then the function/unit of code is said to have a side effect.

Examples of function side effect include modifying any external variable or object property, performing I/O, calling functions having side effects etc.

Total = 0
Fare=[10,20,30,40]
for x in Fare:
    Total += x
mean_fare=Total/len(Fare)
print(mean_fare)

Total = 0

Fare=[10,20,30,40]

for x in Fare:

Total += x

mean_fare=Total/len(Fare)

print(mean_fare)

Let’s see how this code follows the properties of imperative paradigm. First, the order of steps is crucial because if we implement “mean_fare” statement before the loop, it will give a different result. Second, the unit of code i.e. loop, has side effects as it changes the state of the “Total” variable which is outside its scope.

I hope this might have given you the insight of the imperative programming paradigm.

Now, let’s look at some contradictions that I found while searching for this stuff

Some people might argue that the above code belongs to structured programming which is a kind of imperative programming. The main difference between these two is that in the imperative, control flow is defined by the collection of “goto” statements while in the structured, control flow is defined by nested loops, conditionals, and subroutines with no “goto” statements. okay that’s good I said, but then they categorized “Python” into imperative paradigm knowing that it doesn’t have “goto” statement.

So, what I found is that there are terms like structured and non-structured imperative programming. Above code falls into the structured imperative paradigm while if we use the goto statement to define control flow then it falls under the non-structured imperative paradigm.

Declarative Programming paradigm: This is defined as any style of programming that is not imperative. So, the properties will just be the opposite of imperative.

Sequence of the steps is not crucial.
Don’t allow Side effects.

In general, the imperative focuses on “how to do” while declarative focuses on “what to do”meaning logic of software without actually describing its flow. Let’s look at the code below to get more insights into this:

def Mean_list(lst):
    return sum(lst)/len(lst)

1 2	def Mean_list(lst): return sum(lst)/len(lst)

Using reduce() and lambda functions

from functools import reduce
def Mean_list(lst):
    return reduce(lambda a, b: a + b, lst) / len(lst)

from functools import reduce

def Mean_list(lst):

return reduce(lambda a, b: a + b, lst) / len(lst)

Note: Since python in no sense is declarative, the above examples are more towards functional side than declarative. But since functional also falls under the declarative umbrella, above examples can serve as a good platform for understanding declarative concepts.

Explanation: In the above examples, we are describing “WHAT” we want to do rather than HOW (we don’t know HOW lambda and reduce are implemented, we also don’t care). We’re not mutating any state. All of the mutations are abstracted inside of “lambda” and “reduce“. It’s also more readable (once you get used to “lambda” and “reduce“, of course).

Now, you might have got some feeling about the “How to do” and “What to do” approach. Later, this will become more clear when we will go through Functional and other paradigms. Till then, Enjoy.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Programming paradigms

1 Reply

In this posts series, we will be looking at the major programming paradigms and then, at last, we will see which paradigm does python follow.

Hi, have you ever wondered how many programming languages are there? Surfing through the net I found out that there are approx. 256 programming languages till now and still people are working. These languages may have their own pros based on code readability, efficiency or these may be good for different purposes.

But there is one similarity between these languages, that they follow some programming paradigm. We can classify these languages based on which programming paradigm they follow(although many people don’t agree as most of the languages include features from several paradigms so, difficult to classify).

A programming paradigm refers to the style of programming, meaning how you are building the structure and elements of the program. (ufff… Don’t panic from this innocent looking definition, for now just remember that this means some model with a set of rules/features ).There are several features that determine a programming paradigm such as modularity, objects, interrupts or events, control flow etc.

Everyone has an opinion on which programming paradigm/coding style is the best. Sometimes, a single style doesn’t truly solve the problem and you may have to combine different styles on the same problem.

The languages that follow a single paradigm are called “Pure” like Haskell while others are multi-paradigm like python, Java etc.

Common programming paradigms can be “roughly” categorized as:

Although this is not the right way to categorize paradigms. Only for understanding, these paradigms are grouped based on some similar properties. You can see the detailed classification given by Peter Van Roy.

In the next posts, we will look at these paradigms one by one. Till then, Enjoy.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Neural Network Architecture Selection Using Genetic Algorithm

9 Replies

In the previous blog, I have discussed the genetic algorithm and one of its application in the neural network (Training a neural network with a genetic algorithm ). In this blog, I have used a genetic algorithm to solve the problem of neural network architecture search.

You can find full code here .

Genetic Algorithm is really helpful if you do not want to waste your time in using brute force trial and error method for selecting hyperparameters. To know more about the genetic algorithm you can read this blog.

In this tutorial, to demonstrate the use of genetic algorithm I have used Snake Game with Deep Learning where it’s been difficult to find out which neural network architecture will give us the best results. So, the genetic algorithm can be used to find out the best network architecture among the number of hyperparameters.

Different values of hyperparameters are used to create an initial population. I have used the following parameters in the genetic algorithm to find the best value for them.

Number of hidden Layers.
Units per hidden layer
Activation function
Network optimizer

def hyperparameters():
    parameters = []
    units_per_layer = [20,25,30,35,40]
    layers = [1, 2, 3, 4, 5,6,7]
    activation = ['relu', 'tanh', 'sigmoid']
    optimizer = ['adam', 'rmsprop']
    parameters.append(units_per_layer)
    parameters.append(layers)
    parameters.append(activation)
    parameters.append(optimizer)
    
    return parameters

def hyperparameters():

parameters = []

units_per_layer = [20,25,30,35,40]

layers = [1, 2, 3, 4, 5,6,7]

activation = ['relu', 'tanh', 'sigmoid']

optimizer = ['adam', 'rmsprop']

parameters.append(units_per_layer)

parameters.append(layers)

parameters.append(activation)

parameters.append(optimizer)

return parameters

Creating Initial Population

Random parameters are used to create the Initial population. For creating the population first, you have to decide population size. Each individual in the population will have four values.

I have taken 20 chromosomes in the population.

def generate_population(size):
    parameters = hyperparameters()
    
    population = []
    i=0   
    while i < size:
        chromosome = [random.choice(parameters[0]), random.choice(parameters[1]),
                    random.choice(parameters[2]), random.choice(parameters[3])]
        if chromosome not in population:
            population.append(chromosome)
            i+=1
    return population

def generate_population(size):

parameters = hyperparameters()

population = []

i=0

while i < size:

chromosome = [random.choice(parameters[0]), random.choice(parameters[1]),

random.choice(parameters[2]), random.choice(parameters[3])]

if chromosome not in population:

population.append(chromosome)

i+=1

return population

Fitness Function

Fitness function can vary as per the need of different genetic algorithms. Here, I have used the average score for different network architectures. Individuals with the highest average score are fittest ones.

Selection

After evaluating each individual in the population, I have selected top 5 fittest individuals from the population. And also selected 3 individuals from the non-top performers. This will keep us away from getting stuck in the local maximum.

Remaining 12 individuals are created from these 8 individuals using Crossover.

Crossover and Mutation

To produce better offspring for the next generation, I have selected two parents randomly from the 8 individuals selected above and generated other 12 individuals.

def children_population(size, population):
    
    child_population = []
    k = 0
    population_size = len(population)
    while k < size :
        i, j = random.sample(range(0, population_size), 2)
        parent1 = population[i]
        parent2 = population[j]
         
        child = new_child(parent1, parent2) 
        if child not in child_population and child not in population:
            child_population.append(child)
            k +=1
            
    for newest_child in child_population:
        population.append(newest_child)
        
    return population

def new_child(parent1, parent2):
    child = []
    parent_size = len(parent1)
    i=0
    while i<parent_size:
        child.append(random.choice([parent1[i],parent2[i]]))
        i += 1
    return child

def children_population(size, population):

child_population = []

k = 0

population_size = len(population)

while k < size :

i, j = random.sample(range(0, population_size), 2)

parent1 = population[i]

parent2 = population[j]

child = new_child(parent1, parent2)

if child not in child_population and child not in population:

child_population.append(child)

k +=1

for newest_child in child_population:

population.append(newest_child)

return population

def new_child(parent1, parent2):

child = []

parent_size = len(parent1)

i=0

while i<parent_size:

child.append(random.choice([parent1[i],parent2[i]]))

i += 1

return child

In certain new children formed, some of their genes can be subjected to a mutation with a low random probability. Mutation is required to maintain some amount of randomness in the genetic algorithm.

def mutation(population):
    parameters = hyperparameters()
    for chromosome in population:
        if random.random() < 0.1 :
            key = random.choice([0,1,2,3])
            parameters = hyperparameters()
            mutate_key = random.choice(parameters[key])
            chromosome[key] = mutate_key
    
    return population

def mutation(population):

parameters = hyperparameters()

for chromosome in population:

if random.random() < 0.1 :

key = random.choice([0,1,2,3])

parameters = hyperparameters()

mutate_key = random.choice(parameters[key])

chromosome[key] = mutate_key

return population

Now we have created all the necessary functions required for a genetic algorithm. Now, we define a model function using keras library. Then we will train this model with different hyperparameters and search for the best using genetic algorithm.

def train_model(parameters):
    units_per_layer = parameters[0]
    layers = parameters[1]
    activation = parameters[2]
    optimizer = parameters[3]

    model = Sequential()
    model.add(Dense(units=9,input_dim=9))

    for _ in range(layers):
        model.add(Dense(units=units_per_layer, activation=activation))
        
    model.add(Dense(units = 3,  activation = 'softmax'))
    return model

population_with_avg_score = []

for chromosome in population:
    model = train_model(chromosome)
    model.compile(loss='mean_squared_error', optimizer=chromosome[3], metrics=['accuracy'])
    model.fit((np.array(training_data_x).reshape(-1,9)),( np.array(training_data_y).reshape(-1,3)), batch_size = 256,epochs= 3)
    max_score, avg_score = run_game_with_ML(model,display,clock)
    chromosome.append(avg_score)
    chromosome.append(max_score)
    population_with_avg_score.append(chromosome)

def train_model(parameters):

units_per_layer = parameters[0]

layers = parameters[1]

activation = parameters[2]

optimizer = parameters[3]

model = Sequential()

model.add(Dense(units=9,input_dim=9))

for _ in range(layers):

model.add(Dense(units=units_per_layer, activation=activation))

model.add(Dense(units = 3, activation = 'softmax'))

return model

population_with_avg_score = []

for chromosome in population:

model = train_model(chromosome)

model.compile(loss='mean_squared_error', optimizer=chromosome[3], metrics=['accuracy'])

model.fit((np.array(training_data_x).reshape(-1,9)),( np.array(training_data_y).reshape(-1,3)), batch_size = 256,epochs= 3)

max_score, avg_score = run_game_with_ML(model,display,clock)

chromosome.append(avg_score)

chromosome.append(max_score)

population_with_avg_score.append(chromosome)

Here, I have used 10 generations and 20 individuals in the population. It can vary according to your need.

Now, you might have got some feeling about how the genetic algorithm can be applied to find neural architecture instead of using the brute-force method. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

TheAILearner

Mastering Artificial Intelligence

Tag Archives: python

Installing Python OpenCV and other libraries

Python If name == “main”

Python Variables and Scope

Like this:

Side Effects in programming

Like this:

Expressions vs Statements

Expression:

Statements:

Functional programming paradigm

Procedural Programming paradigm

Imperative vs Declarative programming paradigm

Like this:

Programming paradigms

Like this:

Neural Network Architecture Selection Using Genetic Algorithm

Creating Initial Population

Fitness Function

Selection

Crossover and Mutation