Monday, 12 February 2024

ML.NET model training and prediction separately

 In ML.NET, we can train and predict using models in separate steps. ML.NET provides a modular approach to machine learning, allowing you to train a model using training data and then use the trained model to make predictions on new data.

1. Model Training:

First, you need to create a machine-learning pipeline to define your model and its training process. This typically involves loading and transforming your training data, choosing an algorithm, training the model, and saving the trained model to a file. Here's an example using a simple scenario:

TrainData.csproj .net console application


    public class HouseData
    {
        public float Size { get; set; }
        public float Price { get; set; }
    }

    public class HousePricePrediction
    {
        [ColumnName("Score")]
        public float Price { get; set; }
    }


// See https://aka.ms/new-console-template for more information
using Microsoft.ML;
using System.Data;
using TrainData.Models;

MLContext mlContext = new MLContext();

// 1. Import or create training data
HouseData[] houseData = {
               new HouseData() { Size = 1.1F, Price = 11.1F },
               new HouseData() { Size = 2.2F, Price = 22.2F },
               new HouseData() { Size = 3.3F, Price = 33.3F },
               new HouseData() { Size = 4.4F, Price = 44.4F } };
IDataView trainingData = mlContext.Data.LoadFromEnumerable(houseData);

// 2. Specify data preparation and model training pipeline
var pipeline = mlContext.Transforms.Concatenate("Features", new[] { "Size" })
    .Append(mlContext.Regression.Trainers.Sdca(labelColumnName: "Price", maximumNumberOfIterations: 100));

// 3. Train model
var model = pipeline.Fit(trainingData);

// 4. Save model
mlContext.Model.Save(model, trainingData.Schema, "model.zip");
Console.WriteLine(mlContext);



2. Model Prediction: 

Now that you have a trained model saved to a file, you can load the model and use it to make predictions on new data:

PredictionApi.csproj .net webapi

  

    
//Program.cs
    
    using Microsoft.Extensions.ML;
using TrainData.Models;

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.

builder.Services.AddControllers();
// Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

builder.Services.AddPredictionEnginePool()
    .FromFile("H:\\Work\\RND\\ML\\MLSplit\\TrainData\\bin\\Debug\\net7.0\\model.zip");

var app = builder.Build();

// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI();
}

app.UseHttpsRedirection();

app.UseAuthorization();

app.MapControllers();

app.Run();


//FlatController.cs

using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.ML;
using TrainData.Models;

namespace PredictionApi.Controllers
{
    [ApiController]
    [Route("[controller]")]
    public class FlatController : ControllerBase
    {
         

        private readonly ILogger _logger;
        public PredictionEnginePool _predictionEnginePool { get; }

        public FlatController(ILogger logger, PredictionEnginePool predictionEnginePool)
        {
            _logger = logger;
            _predictionEnginePool = predictionEnginePool;
        }


        [HttpGet("{size}:float")]
        public string Get(float size)
        {

            var houseData = new HouseData { Size =size};
            var prediction = _predictionEnginePool.Predict(houseData);
            string sentiment = $"{prediction.Price}";

            return sentiment;
        }
    }
}

Tuesday, 9 January 2024

Start coding with AWS CodeWhisperer and Visual Studio

If you want to enjoy coding with AI intelligence but cannot use GitHub Copilot due to licensing and all,  try Amazon CodeWhisperer. It's free for personal use. 

Brief documentation you may find on AWS official

  • https://aws.amazon.com/codewhisperer/
  • https://aws.amazon.com/codewhisperer/resources/#Getting_started/
  • https://docs.aws.amazon.com/toolkit-for-visual-studio/latest/user-guide/builder-id.html
Easy steps to enable CodeWhisperer on Visual Studio

1. Install/Update AWS Toolkit for Visual Studio On Visual Studio using the Extensions tab


2. Check Amazon CodeWishperer and then Sign up

3. Verify you have successfully authenticated to AWS Builder ID

4. Now click on Try CodeWishperer with examples, add some comments and press Alt+c.




5. Enjoy the fun. :-)






Wednesday, 10 May 2023

Read Excel file using DocumentFormat.OpenXml.Spreadsheet

Code Snippet:


using DocumentFormat.OpenXml.Packaging;
using DocumentFormat.OpenXml.Spreadsheet;
using System;
using System.Data;
using System.IO;
using System.Linq;
using System.Text.RegularExpressions;

namespace ConsoleExcelReader
{
    interface IRajkrsExcelReader
    {
        DataTable GetDataTableFromExcel(string fullPath, string sheetName = null);

    }

    public class RajkrsExcelReader : IRajkrsExcelReader
    {
        private string _traceLocation { get; set; }
        private string _requestId { get; set; }
        public RajkrsExcelReader(string traceLocation)
        {
            this._traceLocation = traceLocation;
            this._requestId = Guid.NewGuid().ToString();
        }
        public DataTable GetDataTableFromExcel(string fullPath, string sheetName = null)
        {
            Log(string.Format("Request initiated for - {0}", fullPath));
            DataTable dt = null;

            if (!File.Exists(fullPath))
            {
                Log(string.Format("file not exists - {0}", fullPath));
            }
            else
            {
                Log(string.Format("file exists - {0}", fullPath));



                dt = new DataTable();

                try
                {
                    using (SpreadsheetDocument spreadsheetDocument = SpreadsheetDocument.Open(fullPath, false))
                    {
                        WorkbookPart workbookPart = spreadsheetDocument.WorkbookPart;

                        Sheet sheet = workbookPart.Workbook.Descendants().Where(s => s.Name == sheetName).FirstOrDefault();


                        if (sheet == null)
                        {
                            Log(string.Format("Sheet {0} not exists", sheetName));
                        }
                        else
                        {
                            Log(string.Format("Sheet {0} exists", sheetName));

                            WorksheetPart worksheetPart = workbookPart.GetPartById(sheet.Id.Value) as WorksheetPart;
                            SharedStringTablePart sharedStringTablePart = workbookPart.SharedStringTablePart;


                            var rowIndex = 1;
                            var rows = worksheetPart.Worksheet.Descendants();
                            Log(string.Format("Row count: {0}", rows.Count()));

                            foreach (Row row in rows)
                            {
                                DataRow dataRow = dt.NewRow();

                                foreach (Cell cell in row.Descendants())
                                {

                                    var currentRowIndex = int.Parse(Regex.Match(cell.CellReference.InnerText, @"\d+").Value);
                                    var currentColumn = Regex.Match(cell.CellReference.InnerText, "[A-Za-z]+").Value;

                                    string cellValue = null;
                                    if (cell.CellValue != null)
                                    {
                                        cellValue = cell.CellValue.InnerText;
                                    }


                                    // If the cell has a data type, convert the value accordingly
                                    if (cell.DataType != null)
                                    {
                                        switch (cell.DataType.Value)
                                        {
                                            case CellValues.SharedString:
                                                cellValue = sharedStringTablePart.SharedStringTable.ElementAt(int.Parse(cellValue)).InnerText;
                                                break;
                                            case CellValues.Boolean:
                                                cellValue = cellValue == "1" ? "TRUE" : "FALSE";
                                                break;
                                            case CellValues.Date:
                                                cellValue = DateTime.FromOADate(double.Parse(cellValue)).ToString();
                                                break;
                                            default:
                                                break;
                                        }
                                    }

                                    if (rowIndex == 1)
                                    {
                                        rowIndex = currentRowIndex;
                                        dt.Columns.Add(new DataColumn(currentColumn, typeof(string)));
                                        dataRow[currentColumn] = cellValue;

                                    }
                                    else
                                    {
                                        dataRow[currentColumn] = cellValue;
                                    }

                                }

                                dt.Rows.Add(dataRow);


                                rowIndex++;
                            }

                            Log(string.Format("Total DUMP available in DT: {0}", dt.Rows.Count));

                            var headerRow = dt.Rows[0].ItemArray;

                            Log("Renaming headers..");
                            for (int i = 0; i < headerRow.Length; i++)
                            {
                                var newHeader = headerRow[i].ToString();
                                Log(string.Format("{0} - {1}", dt.Columns[i].ColumnName, newHeader));
                                dt.Columns[i].ColumnName = newHeader;
                                dt.AcceptChanges();

                            }
                            dt.Rows[0].Delete();
                            dt.AcceptChanges();

                            Log(string.Format("Changes accepted, total {0} rows available.", dt.Rows.Count));
                        }


                    }


                }
                catch (Exception ex)
                {
                    Log(string.Format("Error Message - {0}", ex.Message));
                    if (ex.InnerException != null)
                    {
                        Log(string.Format("Error InnerException - {0}", ex.InnerException.ToString()));
                    }

                }




            }
            return dt;
        }





        private void Log(string msg)
        {
            if (!string.IsNullOrEmpty(_traceLocation))
            {

                try
                {
                    var logFile = _traceLocation + @"Logs\DctExcelReader-trace-" + DateTime.Now.ToString("yyyyMMdd") + ".txt";
                    var dir = Path.GetDirectoryName(logFile);
                    if (!File.Exists(logFile))
                    {
                        if (!Directory.Exists(dir))
                            Directory.CreateDirectory(dir);
                        File.WriteAllText(logFile, "----DctExcelReader Logs----");
                    }
                    File.AppendAllText(logFile, string.Format("\n{0} {1} - {2}", _requestId, DateTime.Now, msg));

                }
                catch (Exception ex)
                {
                    string message = "ExcelReader:" + ex;
                    System.Diagnostics.EventLog.WriteEntry("Application", message, System.Diagnostics.EventLogEntryType.Error, 55555);
                }

            }
        }
    }

}

Thursday, 27 April 2023

How to create .exe in Python?

To create an executable file (.exe) from a Python script, you can use a library called PyInstaller. PyInstaller is a third-party module that can bundle Python scripts into standalone executables.

Here are the steps to create an executable from a Python script using PyInstaller:

Install PyInstaller: You can install PyInstaller using pip, the package manager for Python. Open a terminal/command prompt and enter the following command:

pip install pyinstaller

Create a Python script: Write the Python code that you want to convert to an executable file. Save the script as a .py file.

Generate the executable: Once you have your Python script ready, you can use PyInstaller to generate the executable file. Open a terminal/command prompt, navigate to the directory where your script is located, and enter the following command:

pyinstaller --onefile main.py

Replace "your_script_name.py" with the actual name of your Python script.

The --onefile option tells PyInstaller to create a single executable file, as opposed to a folder containing the executable and other files.

Locate the executable: PyInstaller will create a folder named dist in the same directory as your Python script. Inside the dist folder, you will find the executable file.

That's it! You have successfully created an executable file from your Python script using PyInstaller.

Thursday, 20 April 2023

How to call Rest Api in Python?

To call a REST API in Python, you can use the requests library, a popular library for making HTTP requests. Here's an example of how to use it to make a GET request to a REST API:

import requests


response = requests.get('https://jsonplaceholder.typicode.com/posts/1')


if response.status_code == 200:

    data = response.json()

    print(data)

else:

    print('Error:', response.status_code)



'[ssl: certificate_verify_failed] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)'

pip install python-certifi-win32  if you are getting any SSL certificate error 

or supply verify=False in as 

response = requests.get('https://jsonplaceholder.typicode.com/posts/1', verify = False)


In this example, we're making a GET request to the jsonplaceholder REST API to retrieve a post with ID 1. We first check the response status code to ensure the request was successful and then use the json() method to extract the response data as a Python dictionary.


To make other types of requests, such as POST, PUT, or DELETE, you can use the corresponding methods in the requests library:


import requests



# Make a POST request with JSON data

data = {'name': 'John', 'age': 30}

response = requests.post('https://example.com/api/users', json=data)



# Make a PUT request with form data

data = {'name': 'John', 'age': 35}

response = requests.put('https://example.com/api/users/1', data=data)



# Make a DELETE request

response = requests.delete('https://example.com/api/users/1')


In these examples, we're making a POST request with JSON data, a PUT request with form data, and a DELETE request. Note that the json parameter is used to specify the request data as a JSON object, while the data parameter is used for form data.

How to create Python package locally and use in different project?

To create a Python package locally and use it in different projects, you can follow these steps:

  1. Create a new directory for your package. For example, you can name it mypackage.
  2. Inside the mypackage directory, create a file called __init__.py. This file tells Python that this directory is a package.
  3. Create your Python module(s) in the mypackage directory. For example, you can create a module called mymodule.py.
  4. Inside your module, define your classes and functions.
  5. Create a setup.py file in the mypackage directory. This file contains the package's metadata and tells Python how to install the package. Here is an example of what your setup.py file could look like:
from setuptools import setup, find_packages

setup(
    name='mypackage',
    version='0.1',
    packages=find_packages(),
    install_requires=[
        # list any dependencies your package has
    ],
)

6. Install your package locally by running python setup.py install.

7. Now you can use your package in any other Python project by importing it like this:

import mypackage
from mypackage import mymodule
That's it! Your package is now ready to be used in any Python project. You can continue to add new modules or update your existing modules in your mypackage directory, and then reinstall your package using python setup.py install to use the updated code in other projects.

Monday, 10 April 2023

Write a python program to demonstrate inheritance

 class Animal:

    name = ''

    type = 'carnivorous'


    def base_details(self):

        print(F'{self.name} & {self.type}')



class Dog(Animal):

    def features(self):

        self.name = "Puppy, Gradion, Tommy & jacob etc"

        features = "The dog is a mammal with sharp teeth, an excellent sense of smell, and a fine sense of hearing."

        print(f'Best friend of Human. Lovely names are {self.name}\nIt is {self.type}.\n{features}')



class Cat(Animal):

    def print_type(self):

        self.name = "Cat or Pussy"

        self.type = "herbivorous"

        print(f'{self.name} is a {self.type}.')



dog = Dog()

dog.features()


cat = Cat()

cat.print_type()


Output