Category Archives: Python

Argparse and command line arguments in Python

Leave a reply

In this blog, we will learn what are the command line arguments, why do we use them and how to use them. We will also discuss argparse, the recommended command-line parsing module in the Python standard library.

Let’s start with a very simple example that calculates the area of a rectangle. Let’s name it rect.py.

Running the above script we get

Problem:

Now, if we were to calculate the area of a rectangle for different length and width, we have to change the function arguments. The above code is small so one might not bother changing 2 numbers. But the problem arises when the code is large and one need to change values at multiple locations.

Solution:

This problem can be solved if we provide the arguments at the runtime. For example, what we want is something as shown below

We want the 2’s should somehow be assigned to the length and width variable through the command line so that we don’t need to change the script every time. This can be done easily using the Python argparse module.

Before discussing the argparse module, I hope now you are able to answer the 2 questions, what are the command line arguments and why do we use them?

So, the parameters/arguments given to a program at runtime (For example, the 2’s from the command line shown above) are known as the command-line arguments.

Using command line arguments we can give our program different inputs or additional information without changing the code.

Now let’s see how to parse and access the command line argument. For this, we will use the argparse module.

Let’s again take the above rect.py and add few lines (Line 3 to Line 6)to it as shown below

Before running any program from the command line, the best practice is to know about the program. And this can be done using the -h or –help flag as shown below

So, using the -h or –help flag, we got a overview about the program without even looking into the program.

So, let’s discuss what the code lines we added above actually mean. And how to parse and assign the command-line arguments to length and width.

On Line 3, we instantiate the ArgumentParser object as ap. We can think of it as a container to hold the arguments. The description tells the main purpose of the program(It is optional).

Then on Line 4 and 5, we add the optional arguments length and width. We specify both the shorthand (-l) and longhand versions (–length). To run a program from the command line, specify the flag(either shorthand or longhand or mix) followed by the value as shown below.

The required=True make the optional argument compulsory. For example, if we run rect.py without the width argument it will throw an error as shown below

If we omit writing required=True, None value will be assigned to that flag. For example, omit required=True from the width(Line 5) and re-run rect.py without the width argument, None will be assigned to the width flag.

By default, the arguments are treated as a string so we explicitly defined the type as int for our problem. The help string gives additional information about the argument.

On Line 6, we parse the command line arguments using parse_args() method. This returns an object with the attributes specified in the add_argument() method. The name of the attribute is specified by the dest argument in the add_argument() method. For example in rect.py, ap.parse_args() returns an object with 2 attributes Length and Width.

To access the value of the command line arguments, we can simply use args.attribute_name, where attribute_name is the name specified in the dest argument. For example, args.Length and args.Width in rect.py example as shown above.

So, this is how you can use the argparse module for parsing command line arguments. I hope you now will feel comfortable when dealing with the command line arguments.

This blog contains only a gentle introduction to the argparse module. If you want to explore more, Python documentation is the way to go.

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Floating Point Arithmetic Limitations in Python

Leave a reply

Before moving forward just to clarify that the floating point arithmetic issue is not particular to Python. Almost all languages like C, C++, Java etc. often won’t display the exact decimal number you expect.

First we will discuss what are the floating point arithmetic limitations. Then we will see why this happens and what we can do to compensate these.

I hope you all have some idea of different base representation like 10-base, 2-base etc. Let’s get started

To understand the problem, let’s look at below expressions

Let’s take the first expression and understand why this happens

We know that Computers operate in binary (base 2 fractions). For example, 0.125 decimal fraction is represented as 0.001 in the computer.

There is no problem with integers as we can exactly represent these in binary. But unfortunately, most decimal fractions can’t be represented exactly as binary fractions.

Whenever we divide by a number which is prime and isn’t a factor of base, we always have non-terminating representation.

Thus for base 10, any x=p/q where q has prime factors other than 2 or 5 will have a non-terminating representation.

Simplest example is the case of representing 1/3 in decimal(0.3, 0.33,…, 0.33333…). No matter how many bits (or 3’s) we use, we still can’t represent it exactly.

Similarly, 0.1 is the simplest and the most commonly used example of an exact decimal number which can’t be represented exactly in binary. In base 2, 1/10 is the infinitely repeating fraction of 0011 as shown below

0.0001100110011001100110011001100110011001100110011…

So, Python strives to convert 0.1 to the closest fraction using 53 bits of precision (IEEE-754 double precision).

Thus when we write decimal 0.1 it is stored in computer as not exact but some close approximation. Since 0.1 is not exactly 1/10, summing three values of 0.1 may not yield exactly 0.3, either. Thus

But this contradicts our second expression mentioned above i.e.

This is because of IEEE-754 double precision which uses 53 bits of precision. As the number gets larger it uses more bits. In an attempt to fit into this 53-bit limit, bits have to be dropped off the end which causes rounding and sometimes the error may be visible and sometimes not.

For use cases which require exact decimal representation, you can use decimal or fraction module, scipy or even rounding can help.

Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

PEPs

Leave a reply

PEP stands for Python Enhancement Proposals. According to Python.org
 “A PEP is a design document providing information to the Python community, or describing a new feature for Python or its processes or environment. The PEP should provide a concise technical specification of the feature and a rationale for the feature.”

Anyone can submit their own pep which then will be thoroughly peer-reviewed by the community.

PEP numbers like PEP0, PEP8 etc are assigned by the PEP editors, and once assigned are never changed. (See here for complete pep list)

According to PEP 1, there are three different types of PEPs:

  • Standards: Describes a new feature or implementation.
  • Informational: Tells us about general guidelines or information to the community but doesn’t propose a new feature.
  • Process: Describes a process surrounding Python like procedures, guidelines etc. Unlike informational PEPs, you are not free to ignore them.

There are few PEPs which are worth reading like

  • PEP 8: a style guide for python.
  • PEP 20: The Zen of Python (A list of 19 statements that briefly explain the philosophy behind Python).
  • PEP 257: Docstring Convention.

So, if you see any discrepancy write your PEP and wait for its review. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Python Closures

Leave a reply

Python Closures are related to nested functions(a function defined inside another function). As we have discussed in the previous blog, nested functions can access the variables from the enclosing scope that can be modified using the nonlocal keyword.

You cannot call these variables outside their scopes but closure remembers these values even if they are not present in the memory.

Now, let’s see how to define a closure

Here, the enclosing function returns the nested function instead of calling it. This returned function was bound to name_closure. On calling name_closure(), the name was still remembered although we had already finished executing the enclosing function.

This can be used for data hiding and a substitute for classes if we have only one method in the class (Always prefer using class over this according to me).

You can check for closure function with the help of __closure__ attribute. This returns a tuple of cell objects if it is a closure function otherwise prints nothing as shown below

In the next blog, we will learn decorators that use closures as well. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Python Generators and Yield

Leave a reply

In the previous blog, we have seen how we can build an iterator using __iter__() and __next__() method. In this blog, we will learn an easy and fast way to make our own iterator.

Generators return an iterator object in a simple fashion. Let’s see how we can make generators in Python:

Generator Function: This is the same as the normal function. The only difference is that we use yield statement instead of return. The return statement terminates the function while the yield statement pauses the function and starts from that point again whenever called. Let’s see by an example

The output looks like this

The Odd() function used above is of type generator and ‘a’ is a generator object.

Calling yield, saves all states(x in above code) and continues from that point on calling again ( next(a) returns 3). To iterate again we have to create a new generator object.

We can also use generators with for loop directly.

So, whenever we use yield instead of return, we end up making a generator function.

Generator Expression: This is similar to the list comprehension. Only replace the square bracket with the round. Let’s see an example

Pros: The generator expression gives one element at a time(Lazy evaluation), unlike list comprehensions that output the entire list. Thus, generator expressions are more memory efficient than list comprehensions.

To find the next element in generator expression, use the next() function. Let’s take above code

So, you have learned 3 methods to build an iterator namely:

  1. using __iter__() and__next__() methods(See Python Iterators)
  2. Generator functions
  3. Generator expressions

Using Generators to create an iterator is a fast, easy way but if you want to add extra methods to the iterator object, better switch to the first method.

Now, you might have got some feeling about the Python Generators and yield. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

 

Python Iterators

Leave a reply

Iterators, this name might be new for most of us, but What if I tell you that you have already used this within for loops, comprehensions etc. So, in this blog, let’s see what are iterators, how they work and how we can build our own iterator.

What is an iterator and how it works?

To understand this, we first need to know what is an iterable

Iterable: It is anything that you are able to loop over like lists, tuples, strings etc.

Iterator: It is the object that does the actual iterating.

So, in order to iterate, you must first convert an iterable into an iterator. This is done by using built-in iter() function.

To get the next element, use built-in next() function on that iterator. next() function raises a StopIteration exception when there are no more elements in that iterator.

Note: If we directly apply next() on iterable like lists etc. it will give a typeerror as

Thus, __iter__() and __ next__() (called the iterator protocol)are the two special methods implemented by the iterator.

Pros:  Iterators are lazy, meaning, they don’t determine what their next item is until you ask them for it. This saves memory, time and can work with infinite sequences.

Building our own iterator:

To build an iterator, we only need the methods __iter__() and __next__(). Below, we show an example that will give us the next Fibonacci term in each iteration.

Now we can create an iterator and iterate through it as follows.

Here, I haven’t called the iter() method because it returns the object itself. You may call this, as this will not affect anything.

We can also use for loop to iterate over our Fib Class as:

Here, for loop doesn’t give stopiteration exception because it uses try-except statement(See here how the for loop works)

Now, you might have got some feeling about the Python iterators and iterable. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

What’s inside Python ‘for’ loop?

2 Replies

Have you ever wondered how ‘for’ loop is automatically able to iterate over iterable like lists, strings etc? What if I tell you that Python for loop is actually an infinite while loop. Most of us might not believe but as is this beautiful Python.

There must be something inside ‘for’ loop that helps it to iterate over any iterable. In this blog, we will see how the for loop is actually implemented in python. Let’s understand this with an example.

Suppose we want to sum the elements of a list [1,2,3,4]. This can be done as

Actually what happens inside for loop is this

To understand this, we first need to know what iterable and iterators are

  • Iterable: It is anything that you are able to loop over like lists, tuples, strings etc.
  • Iterator: It is the object that does the actual iterating.

So, in order to iterate, you must first convert an iterable into an iterator. This is done by using built-in iter() function. To get the next item, use next() function on that iterator (not on iterable). If there are 100 elements in an iterable then we need to call next() 100 times. To avoid this, we use an infinite while loop. next() function raises a StopIteration exception when there are no more elements in that iterable. So, to avoid this exception, we have used try-except statement and in this way, we break out from the infinite loop.

This is how the for loop actually works in python.

Now, you might have got some feeling about the Python for loop, iterator and iterable. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Python If __name__ == “__main__”

1 Reply

There are two things which we can do with our script/program, either we run it directly or import it into another program. Suppose we have a my_module.py file, as shown below

To run the my_module.py file directly we write python my_module.py on the command line. But if we want to import this file, then we have to comment out the function() calling statement. Import file means we only want functions, classes etc. and no code execution statements like a function call from that file. Just let the user call these at his own will.

For example, import math only imports functions from the math.py file. It is up to us whether we call math.sin() or math.cos(). That’s why we should remove code execution statements before importing a file.

Problem: “Commenting out function call from the file which we want to import”. For large modules, this is seriously a headache.

Question: Can’t we do something to automate the above problem, meaning writing import should automatically comment out or remove the function call.

Solution: Python if __name__ == “__main__” statement.

Concept: Before executing the code, Python defines a few special variables. For example, if the Python interpreter is running that module (the source file) directly, it sets the special __name__ variable to have a value “__main__”. If this file is being imported from another module, __name__ will be set to the module’s or file’s name.

So, what we do is write all the code execution statements inside if __name__ == “__main__” statement. When we run the file directly this condition is satisfied and the code is executed. While importing __name__ == file_name and not __main__  so the code is not executed.  Let’s see an example

Running the above hello.py file directly (F5 in shell or by command line), we get this output

Now, import this hello.py into another file say hi.py like as shown below

Running the above code gives the output shown below

Clearly, the func() is not executed while importing and this is what we want.

Now, you might have got some feeling about the Python if __name__ == “__main__” statement. Always use this. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Python Variables and Scope

1 Reply

To understand what is happening in the code, you should know what the “scope” of variable means. The scope of a variable refers to the places from where we can see or access a variable. To understand this more clearly, we should first know what “variable” actually means.

A variable is a reference to a memory address that contains the value. Let’s see the figure below.

Suppose, we have x=5. There are two ways of reading this line.

  1. ‘x’ is a variable name that references to a memory location in which 5 is stored.
  2. variable x equals 5

Saying the humungous first line every time may be frustrating, so we usually use the second way. Here comes the confusion. Let’s see by an example

Question: How many variables are there in the above code?

If we use the definition x equals 5, we might think that it is the same variable x with different values. But if we use the actual definition of the variable, we can argue that there are 3 variables as they refer to different memory locations and thus are different. We can always check the identity of the location of the object in memory using the python id() function. Just write id(variable_name) which outputs a number that refers to some memory location(same number means the same variable). So, use the shortcut definition but always remember the actual meaning.

This means we can define a number of variables with the same name(say x) in a program without having a relation between them. This sometimes creates confusion. Let’s understand this by an example

Now, which value should this x prints? This is where the scope comes into the picture. Scope tells us which variable we can access and from where.

LEGB Rule: LEGB stands for Local, Enclosing, Global and Built-in. This tells us the order in which python accesses the variable in a program. First, python will check if the variable is present in the local scope, if not then checks in enclosing scope and so on till the builtin scope is checked.

Let’s understand the LEGB terms one by one:

  1. Local: Python first tries to search for an identifier(variable) in Local scope. The local variable exists only within the block/function that it is declared in. When that block/function ends, the local variable has no significance, it gets destroyed. We cannot use it outside the function where it is declared. For example in the code below, x=3 is local to the number() function means we cannot access that variable outside this function. So, the output of the last print(x) will be an error saying name ‘x’ is not defined.
  2. Enclosing: Enclosing refers to the nested functions where one encloses the other. If Python does not find an identifier(variable) within the local scope, it will examine the Enclosing scope to see if it can find the variable there. Let’s understand this by an example

    For the print x —1, inner() is Local and outer is Enclosing while for the print x—2, outer() is Local. So, print x—1 will first search for x in the inner(). Since there is no x in the inner() so we go to outer()/Enclosing and check if x is there or not. So, print x—1 outputs 4. For print x–2, we have x in the Local scope of outer() so this will output 4.
  3. Global: Global variable means that it is accessible from anywhere in your script, including from within a function. It is usually defined at the top of the script or outside of the function. If Python fails to find the variable in both above scopes, then it will check the global scope. Let’s see an example

    As we have no x in the local and enclosing scope, so Python checks the global scope and outputs 4 for both print x statements.
    Note: using too many global variables will generally lead to name conflicts and will likely lead to unwanted side effects.
  4. Built-in: Built-in means the function and variables that are already defined in python. If an identifier is not found in any of the above scopes within a module, then Python will examine the built-in identifiers to see if it is defined there. For example

    If len is not found in any of the scopes, then Python would consult the Built-In scope, where it will find the len function and outputs 6. But if we create our own len function then it will have precedence over Built-in len function. Let’s see by an example

    Since our len function precedes over built-in len and thus the output of print x statement equals 1.

Now, I hope that you might have understood the Python variable and scope, Let’s look at some of the keywords that are useful regarding this

  1. Global Keyword: With this, we tell the Python to use the globally defined variable instead of locally defining it. This is useful when we want to change the value of a variable through a function. To use it, simply type global followed by the variable name. Let’s see an example
  2.  NonLocal Keyword: This is useful in nested functions. It causes the variable to refer to the previously bound variable in the closest enclosing scope. Let’s see by an example

    By nonlocal, we tell Python that this x is same as defined in the enclosing outer() function.

Now, you might have got some feeling about the Python variable and scope. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.

Side Effects in programming

Leave a reply

According to Wikipedia, a function or expression is said to have a side effect if it modifies some state outside its local environment. Let’s understand this in depth by examples.

To understand side effects, first, we should know what “state” is. In layman terms, the state refers to the value of the variable at any given point in the program like x=2.

Now, the above Wikipedia definition might have started to make some sense. If not, don’t worry, see the figure below.

In this figure, we have a variable x that is outside the scope of two functions. Clearly, after the Function_2 call, the value of x changes which means that Function_2, other than returning value(if any), is interacting with variables outside its local environment and thus has a side effect.

In general, any lasting effect that occurs in a function, not through its return value, is called a side effect. Not through its return value means if we assign x=Function_1(), this will clearly change the value of x but this is not a side effect. For side effect, the change should occur inside a function.

Common ways to have side effects:

  1. Modifying a non-local variable: Let’s understand by an example

    The x in the square() function is local to its scope so no change in outside x. Global in square_side_effect() makes x global so any change in x will change it in whole code (outside function scope) so the square_side_effect function() is not pure(has side effects).
  2. Changing the value of a mutable object: Let’s take Lists which is a mutable object.

    Clearly, after calling list_side_effect(), the value of lst changes so, list_side has side effects.
  3. Performing Input/Output operations: This doesn’t change any objects or variable value, but it does have a potential lasting effect outside the function call because a person might see the output and be influenced by it.
  4. Calling other side-effect functions

Now, you might have got some feeling about the side effects. Hope you enjoy reading.

If you have any doubt/suggestion please feel free to ask and I will do my best to help or improve myself. Good-bye until next time.