8 Python

8.0.1 Map and Filter

Map is used to apply a function to a list.

It computes my_square for all items in seq.

Lambda

This is used to define a function in a line.

lambda num: num*2

That means passing variable : return statement.

Useful for use in map.

8.0.1.1 Filters

Filter is used to filter items in seq based on function/lambda

eg: to get even values

filter(lambda num: num%2 == 0,seq)

8.1 Web Dev Libs

Flask

  • light weight web framework
  • FlaskRESTful - extention for REST framework

sqlite3

  • store to local db

Shelve

  • Persistant object storage library

Pickle

  • Dump objects to file and load back

Pandas

  • Data maipulation
  • Pandas vs Dask vs Spark
    • less than a gb, use pandas
    • upto 100 gb, use pandas with chunk or dask or pyspark
    • more than 100gb, pyspark for sure, upto peta bytes
    • Pandas, Dask or PySpark - Medium
  • Superfast Pandas, make data transformation in pandas upto 7237 times faster:
    • use iterrow
    • use apply(lambda)
    • use pandas vectorization, loc(filters here)
    • use numpy vectorization, pass as numpy.array, converts to C code.
    • Making pandas loop faster