Tokens In Python

Tokens in Python -:

Today we will discuss "Python tokens", "Types of tokens with example", "List of tokens in Python", "How many types of tokens are allowed in Python".The Smallest individual unit in a python program is known as "Tokens".Python has mainly 5 types of token which are given below -:

1. Keywords
2. Identifiers
3. Operators
4. Literals 
5. Punctuators 

Keywords in Python

What are the keywords in Python??

Keywords are the reserved word having special meaning and purpose in Python. Keywords cannot be used as an identifier name such as any variable name or Function Name. They are used to defined "Syntax" or "Structure" of the Python language. Python Keywords are case sensitive.

Python keywords is a special word, that forms the vocabulary of the python language. It is a reserved word that can not be used as an identifier.

How many keywords are there in Python??

There are as many as 33 keywords are used in Python programming language version 3.7. Keyword builds the vocabulary of the python language, they represent the "syntax and structure" of a python program.

How do you find a keyword in Python??

Following is the list of keywords in the python, there are 33 Keywords:

S No	Keywords	Description
1	and	A logical operator
2	as	To create an alias
3	assert	For debugging
4	break	To break out of a loop
5	class	To define a class
6	continue	It skips the remaining part of the loop and goes to the next iteration
7	def	To define a function
8	del	To delete an object
9	elif	Used in conditional statements, same as else if
10	else	Used in conditional statements
11	except	It is used with exceptions. this part will execute if an exception occurs.
12	FALSE	Boolean value, the result of comparison operations
13	finally	It is used with exceptions. It executes irrespective of exception occurs or not
14	for	To create a for loop
15	from	To import specific parts of a module
16	global	To declare a global variable
17	if	To make a conditional statement
18	import	To import a module
19	in	To check if a value is present in a list, tuple, etc.
20	is	To test if two variables are equal
21	lambda	To create an anonymous function
22	None	Represents a null value
23	nonlocal	To declare a non-local variable
24	not	A logical operator
25	or	A logical operator
26	pass	It is a "do nothing" statement.
27	raise	To raise an exception
28	return	To exit a function and return a value
29	TRUE	Boolean value, result of comparison operations
30	try	To make a try...except statement
31	while	To create a while loop
32	with	Used to simplify exception handling
33	yield	To end a function returns a generator

  

Identifier in Python

Before knowing Python Identifiers, we will learn what is Identity?? 

Identity is a property that makes a person different, it may be personality looks or expressions. Same way, Python identifier is the same as the identity of a Person.

Names given to a Class, Function or Variable is called Identifier in python. It helps to differentiate one entity from others.

What are the rules for identifiers??

1. Identifier first letter should be any letter or underscore(_).
2. Upper and lower-case letters are different.
3. The digit 0 through 9  can be part of the identifier except for the first character.
4. It should not be a keyword (list of 32 keywords discussed above).
5. An identifier can not contain any special character except for underscore (_).

Valid / Invalid  Identifiers Examples -:

Valid Identifier -:  Mybook, file 123, z2td, _no

Invalid Identifier  -: 2rno , break , .mybook

Let us discuss why the given identifiers are Invalid.

• 2rno: It is starting with a number
• break: It is a Keyword
• .mybook: It is containing "." keyword.  

Operators in Python

Operators are tokens that trigger some computation/action when applies to variables and other objects in an expression. Variables and objects to which the computation is applied are called operands.

There are following types of Operators:

1. Arithmetic Operators
2. Relational Operators
3. Assignment Operators
4. Logical Operators
5. Bitwise operators
6. Membership Operators
7. Identity Operators

Arithmetic Operators -:

• Arithmetic Operators are used to performing arithmetic operations like Addition, Subtraction, Multiplication, division etc.
• There are following types of Arithmetic Operators in Python:

  Operators	Description	Example
  +	perform addition of two number	a+b
  -	perform subtraction of two number	a-b
  /	perform division of two number	a/b
  *	perform multiplication of two number	a*b
  %	Modulus = returns remainder	a%b

Relational (Comparison )Operators -:

• Relationa operators perform an operation on data and return the result in Boolean true or False.
• Relational Operators are used to comparing the value and decide the relation among them.

Operators	Description	Example
==	True if a is equal to b	a == b
!=	True is a is not equal to b, otherwise False	a != b
>	True if a is greater than b, otherwise False	a>b
>=	Greater than or equal to, return true if a is greater than b or a is equals to b	a>=b
<	True if a is less than b, otherwise False	a < b
<=	Less than or equal to, return true if a is less than b or a is equals to b	a<=

Assignment Operators -:

• Assignment Operators are used to assigning value to a variable.
• There are following types of assignment operators are there in Python:

Operators	Description	Example
=	Right side value will be assigned to the left side.	a=b
+=	both the operands will be added and result will be assigned to the left side operand.	a+=b
/=	left operand divided by right operand, the result will be assigned to the left side operand	a/=b
*=	both the operands will be multiplied and the result will be assigned to the left side operand.	A*=b
-=	left operand minus right operand, the result will be assigned to the left side operand	A-=b
%=	left operand modulus right operand, the result will be assigned to the left side operand	a%=b
//=	left operand floor divide right operand, the result will be assigned to the left side operand	a//=b
**=	the left operand to the power of right operand, the result will be assigned to the left side operand.	a**=b

Logical Operators -:

• Logical operators are used with conditional statements that can be TRUE or False.
• Logical Operators are used to performing logical operations on the two given variables or values.
• and, or and not are logical operators.

Operators	Description	Example
and	return true if both conditions are true	x and y
or	return true if either or both condition is true	x or y
not	reverse the condition	not(a>b)

Bitwise Operators -:

• Bitwise Operators acts on operands as if they were string or binary digits. They operate bit by bit hence its name is bitwise Operator.

Operator	Meaning	Example
&	Bitwise AND	x & y = 0 (0000 0000)
|	Bitwise OR	x | y = 14 (0000 1110)
~	Bitwise NOT	~x = -11 (1111 0101)
^	Bitwise XOR	x ^ y = 14 (0000 1110)
>>	Bitwise right shift	x >> 2 = 2 (0000 0010)
<<	Bitwise left shift	x << 2 = 40 (0010 1000)

Membership Operators -:

• The Membership operators are used to validate whether a value is found within a sequence such as string, lists or tuples.



Operators	Description	Example
in	return true if a value exists in the sequence, else false.	a in list
not in	return true if the value does not exist in the sequence, else false.	a not in list

Identity Operators -:

• Identity operators are used to comparing the memory location of two objects. 

  Operators	Description	Example
  is	returns true if two variables point the same object, else false	a is b
  is not	returns true if two variables point the different object, else false	a is not b

Literal (Value) in Python

• Literals like a constant (often referred to as constant - values) are data items that have a fixed value.
• Literals yields an object of the given type (String, integer, Long integer, Long floating number, Complex number ) with the given value.
  Python allows several kinds of literals.
  1. String Literals
  2. Numeric Literals
  3. Boolean Literals.

SN	Literal Name	Description
1	String Literal	A string literal is a sequence of characters surrounded by quotes (single or double or triple quotes) Example-: 'a' , 'abc' , "abc"
2	Numeric Literal	1. int (signed integers) -: No decimal points. Example -: age=22 2. float (floating point real values) –: represent real number & written a decimal point. Example -: height = 5.2 3. complex (complex numbers) Example-: name = None
3	Boolean Literal	1. true (Boolean Literal) 2. false (Boolean Literal)

Punctuators  in Python

• Punctuators is used to implement the grammar and structure of syntax. 
• Punctuators are symbols that are used in a programming language to organize programming sentence.

The most common punctuators of Python programming language are:-  

|	'	"	#	\	[	]	:	;	=
(	)	{	}	@	<<=	>>=	**=	%=	&=