How to get data from database and renders in template files using transformation method in django

Transformation in Python refers to the process of modifying or converting data from its original form to a desired form. It involves applying operations or functions to manipulate the data and generate a transformed result. Python provides various methods and techniques to perform transformations on data.

Here are some commonly used transformation methods in Python:

List Comprehension: List comprehension is a concise way to create new lists by transforming or filtering existing lists. It allows you to iterate over an iterable (e.g., list, string) and apply an expression or operation to each element. The transformed elements are collected in a new list. For example:

numbers = [1, 2, 3, 4, 5]
squared_numbers = [num ** 2 for num in numbers]

Output:[1, 4, 9, 16, 25]

Mapping Functions: Python provides built-in functions like map() that can be used to apply a function to each element of an iterable and generate a new iterable with the transformed elements. For example:

numbers = [1, 2, 3, 4, 5]
squared_numbers = list(map(lambda x: x ** 2, numbers))

Output:

[1, 4, 9, 16, 25]

Filtering Functions: Python provides built-in functions like filter() that can be used to apply a function to each element of an iterable and generate a new iterable with the elements that satisfy a specified condition. For example:

numbers = [1, 2, 3, 4, 5]
even_numbers = list(filter(lambda x: x % 2 == 0, numbers))

Output

[2, 4]

Combinations of Built-in Functions: You can also combine multiple built-in functions to perform complex transformations. For example, you can use map() and filter() together to apply a function and filter the elements simultaneously. For example:

numbers = [1, 2, 3, 4, 5]
transformed_numbers = list(map(lambda x: x ** 2, filter(lambda x: x % 2 == 0, numbers)))

Output:

[4, 16]

Generator Expressions: Similar to list comprehension, generator expressions allow you to create generators that produce a sequence of values on-the-fly, without storing them in memory. They can be used for efficient transformations of large datasets. For example:

numbers = [1, 2, 3, 4, 5]
squared_numbers = (num ** 2 for num in numbers)

These are some common methods for performing data transformation in Python. Each method has its own advantages and use cases, depending on the specific requirements of

Define the model

In the models.py file of your app, define a model representing the table. For example, let's assume the table is named MyTable with fields name and age.

from django.db import models

class MyTable(models.Model):
    name = models.CharField(max_length=100)
    age = models.IntegerField()

Run migrations to create the table in the database using python manage.py makemigrations followed by python manage.py migrate.
Step 3: Populate the table with sample data (optional)

Open a Django shell using python manage.py shell.
Import the model: from myapp.models import MyTable.
Create some sample records:

MyTable.objects.create(name='John', age=25)
MyTable.objects.create(name='Alice', age=30)

Exit the shell.
Step 4: Views and transformation methods
Now, let's define the views and transformation methods to fetch data from the table and apply transformations.

Option 1: Using raw SQL queries

from django.db import connection
from django.shortcuts import render

def raw_query_view(request):
    with connection.cursor() as cursor:
        cursor.execute("SELECT name, age FROM myapp_mytable")
        rows = cursor.fetchall()
    transformed_data = [(name.upper(), age * 2) for name, age in rows]
    return render(request, 'template.html', {'data': transformed_data})

Output

<ul>
  {% for name, age in data %}
    <li>Name: {{ name }}, Age: {{ age }}</li>
  {% endfor %}
</ul>

Option 2: Using Django ORM

from django.shortcuts import render
from myapp.models import MyTable

def orm_view(request):
    rows = MyTable.objects.all().values('name', 'age')
    transformed_data = [(row['name'].upper(), row['age'] * 2) for row in rows]
    return render(request, 'template.html', {'data': transformed_data})

display in template file

<tbody>
    {% for item in data %}
    <tr>
      <td>{{ item.0 }}</td>
      <td>{{ item.1 }}</td>
    </tr>
    {% endfor %}

Option 3: Using queryset annotations

from django.db.models import F
from django.shortcuts import render
from myapp.models import MyTable

def annotation_view(request):
    rows = MyTable.objects.annotate(upper_name=F('name').upper(), doubled_age=F('age') * 2)
    transformed_data = [(row.upper_name, row.doubled_age) for row in rows]
    return render(request, 'template.html', {'data': transformed_data})

Option 4: Using list comprehension

from django.shortcuts import render
from myapp.models import MyTable

def list_comprehension_view(request):
    rows = MyTable.objects.all()
    transformed_data = [(row.name.upper(), row.age * 2) for row in rows]
    return render(request, 'template.html', {'data': transformed_data})

Option 5: Using pandas library

import pandas as pd
from django.shortcuts import render
from myapp.models import MyTable

def pandas_view(request):
    queryset = MyTable.objects.all().values('name', 'age')
    df = pd.DataFrame.from_records(queryset