WebVeta - Advanced, unified, consistent search for your website(s), from content of your website(s), blogs(s). First 50 customers, who sign-up prior to 15/05/2024 get unlimited access to existing features, newer features for at least 1 year. Sign up now! https://webveta.alightservices.com/
Categories
.Net A.I Azure C# LLM NLP

Using C# and Azure OpenAI services

If you have access to Azure OpenAI services, the following code snippet shows how to chat with ChatGPT!

Some general tips:

  1. Secure your networks, i.e use private endpoints inside Azure! Stolen keys can be used by other people if the network is not secured.

This code example is C# version of what’s discussed in https://github.com/AzureCosmosDB/Azure-OpenAI-Python-Developer-Guide/blob/main/05_Explore_OpenAI_models/README.md

The above link is for Python developers, this blog post for C# developers.

var chatClient = new OpenAIClient(new Uri(azureEndPoint), 
       new AzureKeyCredential(apiKey));

var chatCompletionOptions = new ChatCompletionsOptions();

chatCompletionOptions.DeploymentName = "gpt35";

chatCompletionOptions.Messages.Add(new
    ChatRequestSystemMessage("You are a helpful, fun and friendly sales assistant for Cosmic Works, a bicycle and bicycle accessories store."));

chatCompletionOptions.Messages.Add(new 
    ChatRequestUserMessage("Do you sell bicycles?"));

chatCompletionOptions.Messages.Add(new
    ChatRequestAssistantMessage("Yes, we do sell bicycles. What kind of bicycle are you looking for?"));

chatCompletionOptions.Messages.Add(new
    ChatRequestUserMessage("I'm not sure what I'm looking for. Could you help me decide?"));

var response = await 
    chatClient.GetChatCompletionsAsync(chatCompletionOptions);

if (response != null && response.Value != null && 
    response.Value.Choices != null && 
    response.Value.Choices.Count > 0)
{

    System.Console.WriteLine(
        response.Value.Choices.ElementAt(0).Message.Content);
}

The above code has 3 configuration variables:

  1. azureEndPoint – This is the endpoint from Azure Portal.
  2. apiKey – One of the API keys from Azure Portal.
  3. The deployment name of the model that has been added through Azure Portal.

If the code was run successful, output looks like this:

If there is network connectivity issues, there might be exceptions, if denied due to network security issues errors might be like the following:

I don’t have any fake aliases, nor any virtual aliases like some of the the psycho spy R&AW traitors of India. NOT associated – “ass”, eass, female “es”, “eka”, “ok”, “okay”, “is”, “erra”, yerra, karan, kamalakar, diwakar, kareem, karan, sowmya, zinnabathuni, bojja srinivas (was a friend and batchmate 1998 – 2002), mukesh golla (was a friend and classmate 1998 – 2002), thota veera, uttam’s, bandhavi’s, bhattaru’s, thota’s, bojja’s, bhattaru’s or Arumilli srinivas or Arumilli uttam (may be they are part of a different Arumilli family – not my family).

Mr. Kanti Kalyan Arumilli

Arumilli Kanti Kalyan, Founder & CEO
Arumilli Kanti Kalyan, Founder & CEO

B.Tech, M.B.A

Facebook

LinkedIn

Threads

Instagram

Youtube

Founder & CEO, Lead Full-Stack .Net developer

ALight Technology And Services Limited

ALight Technologies USA Inc

Youtube

Facebook

LinkedIn

Phone / SMS / WhatsApp on the following 3 numbers:

+91-789-362-6688, +1-480-347-6849, +44-07718-273-964

+44-33-3303-1284 (Preferred number if calling from U.K, No WhatsApp)

kantikalyan@gmail.com, kantikalyan@outlook.com, admin@alightservices.com, kantikalyan.arumilli@alightservices.com, KArumilli2020@student.hult.edu, KantiKArumilli@outlook.com and 3 more rarely used email addresses – hardly once or twice a year.

Categories
.Net A.I Artificial Intelligence C# Llama LLM NLP

How to get text embeddings from Meta Llama using C# .Net

This post is about getting text embeddings i.e vector representation of text using C# .Net and using Meta’s Llama 2!

Meta’s Llama

Meta (Facebook) has released few different LLM’s, the latest Llama3, but this blog post about Llama2. Using Llama3 might be similar, but I have not tried yet! There are few more things that can be tried, but those are out of scope and this is an end to end blog post for using Llama2 using C#.

https://llama.meta.com/

From the above link provide click “Download Models”, provide information. Then links to some github, some keys are provided. Make note of the keys. The keys are valid for 24 hours and each model can be downloaded 5 times.

llama.cpp

We use llama.cpp for certain activities:

https://github.com/ggerganov/llama.cpp

LLamaSharp

This is the wrapper for interacting from C# .Net with Llama models.

I have introduced the tools and software that are going to be used. Now, let’s look at the different steps:

  1. Download Llama model (Meta’s Llama has Llama 2 and Llama 3, each has smaller and larger models, this discusses the smallest model from Llama 2)
  2. Prepare and convert Llama model into gguf format.
  3. Use in C# code

Download Llama model:

Once you submit your information and receive the keys from Meta Facebook, clone the repo:

https://github.com/meta-llama/llama for Llama2,

https://github.com/meta-llama/llama3 for Llama3

git clone https://github.com/meta-llama/llama

Navigate into llama folder, then run download.sh

cd llama
sudo ./download.sh

You would be prompted for the download key, enter the key.

Now 12.5 GB file gets downloaded into a folder “llama-2-7b”

Prepare and convert Llama model into gguf format:

We are going to convert the Llama model into gguf format. For this we need Python3 and Python3-Pip, if these are not installed, install using the following command

sudo apt install python3 python3-pip

Clone the llama.cpp repo into a different directory.

git clone https://github.com/ggerganov/llama.cpp

Navigate into llama.cpp and compile

cd llama.cpp
make -j

Install the requirement for python:

python3 -m pip install -r requirements.txt

Now copy the entire “llama-2-7b” into llama.cpp/models.

Listing models directory should show “llama–2-7b”

ls ./models
python3 convert.py models/llama-2-7b/

This generates a 2.17 GB file ggml-model-f32.gguf

Now run the following command:

./quantize ./models/llama-2-7b/ggml-model-f32.gguf ./models/llama-2-7b/ggml-model-Q4_K_M.gguf Q4_K_M

This should generate a 3.79 GB file.

Optional (I have NOT tried this yet)

The following extra params can be passed for the python3 convert.py models/llama-2-7b/

python convert.py models/llama-2-7b/ --vocab-type bpe

C# code

Create a new or in an existing project add the following Nuget packages:

LLamaSharp

LLamaSharp.Backend.Cpu or LLamaSharp.Backend.Cuda11 or 
LLamaSharp.Backend.Cuda12 or LLamaSharp.Backend.OpenCL

// I used LLamaSharp.Backend.Cpu

Use the following using statements:

using LLama;
using LLama.Common;

The following code is adapted from the samples of LlamaSharp – https://github.com/SciSharp/LLamaSharp/blob/master/LLama.Examples/Examples/GetEmbeddings.cs

string modelPath = PATH_TO_GGUF_FILE

var @params = new ModelParams(modelPath) {EmbeddingMode = true };
using var weights = LLamaWeights.LoadFromFile(@params);
var embedder = new LLamaEmbedder(weights, @params);

Use the path for your .gguf from quantize step file’s path.

Here is code for getting embeddings:

float[] embeddings = embedder.GetEmbeddings("Hello, this is sample text for embeddings").Result;

Hope this helps some people, I am .Net developer (primarily C#), A.I enthusiast.

I don’t have any fake aliases, nor any virtual aliases like some of the the psycho spy R&AW traitors of India. NOT associated – “ass”, eass, female “es”, “eka”, “ok”, “okay”, “is”, “erra”, yerra, karan, kamalakar, diwakar, kareem, karan, sowmya, zinnabathuni, bojja srinivas (was a friend and batchmate 1998 – 2002), mukesh golla (was a friend and classmate 1998 – 2002), thota veera, uttam’s, bandhavi’s, bhattaru’s, thota’s, bojja’s, bhattaru’s or Arumilli srinivas or Arumilli uttam (may be they are part of a different Arumilli family – not my family).

Mr. Kanti Kalyan Arumilli

Arumilli Kanti Kalyan, Founder & CEO
Arumilli Kanti Kalyan, Founder & CEO

B.Tech, M.B.A

Facebook

LinkedIn

Threads

Instagram

Youtube

Founder & CEO, Lead Full-Stack .Net developer

ALight Technology And Services Limited

ALight Technologies USA Inc

Youtube

Facebook

LinkedIn

Phone / SMS / WhatsApp on the following 3 numbers:

+91-789-362-6688, +1-480-347-6849, +44-07718-273-964

+44-33-3303-1284 (Preferred number if calling from U.K, No WhatsApp)

kantikalyan@gmail.com, kantikalyan@outlook.com, admin@alightservices.com, kantikalyan.arumilli@alightservices.com, KArumilli2020@student.hult.edu, KantiKArumilli@outlook.com and 3 more rarely used email addresses – hardly once or twice a year.

Categories
.Net C# NLP

C# .Net, Python and NLP – Natural Language Processing

The available NLP libraries for C# are very less. Some of the best NLP libraries are in Python. There are few C# implementations that either are a re-write of Python’s implementations or wrappers.

If you are serious about any NLP related work in C#, use well-maintained Python implementations and integrate with C#. The wrappers / re-write’s might not have much support.

Let’s look at 2 popular libraries:

NLTK – Natural Language Toolkit

spaCy – Industrial-Strength Natural Language Processing

Both the libraries are well documented and easy to use. i.e even C# developer like me can understand the python code examples provided. And very easy to experiment.

Now here are some ways of integrating with C#:

  1. Have a separate set of micro-services or message based applications for NLP.
  2. Use some of the following libraries that attempt to integrate Python with .Net
  3. Use System.Diagnostics.Process class i.e run the Python program as a separate process.

The integration libraries are:

Python.NET

IronPython

Python.Included

I don’t have any fake aliases, nor any virtual aliases like some of the the psycho spy R&AW traitors of India. NOT associated – “ass”, eass, female “es”, “eka”, “ok”, “okay”, “is”, “erra”, yerra, karan, kamalakar, diwakar, kareem, karan, sowmya, zinnabathuni, bojja srinivas (was a friend and batchmate 1998 – 2002), mukesh golla (was a friend and classmate 1998 – 2002), thota veera, uttam’s, bandhavi’s, bhattaru’s, thota’s, bojja’s, bhattaru’s or Arumilli srinivas or Arumilli uttam (may be they are part of a different Arumilli family – not my family).

Mr. Kanti Kalyan Arumilli

Arumilli Kanti Kalyan, Founder & CEO
Arumilli Kanti Kalyan, Founder & CEO

B.Tech, M.B.A

Facebook

LinkedIn

Threads

Instagram

Youtube

Founder & CEO, Lead Full-Stack .Net developer

ALight Technology And Services Limited

ALight Technologies USA Inc

Youtube

Facebook

LinkedIn

Phone / SMS / WhatsApp on the following 3 numbers:

+91-789-362-6688, +1-480-347-6849, +44-07718-273-964

+44-33-3303-1284 (Preferred number if calling from U.K, No WhatsApp)

kantikalyan@gmail.com, kantikalyan@outlook.com, admin@alightservices.com, kantikalyan.arumilli@alightservices.com, KArumilli2020@student.hult.edu, KantiKArumilli@outlook.com and 3 more rarely used email addresses – hardly once or twice a year.

Categories
.Net Azure C# Key Vault

How to add secrets into Azure KeyVault using C#

Azure KeyVault is a Azure service for secrets management. Secrets can be easily added, updated, retrieved etc… This post has small code snippet, code implementation can be found here:

https://github.com/ALightTechnologyAndServicesLimited/LightKeysTransfer/blob/main/LightKeysTransfer/LightKeysTransfer/Implementation/AzureKeyVaultHelper.cs

The above code is from my own open source project – LightKeysTransfer, the code can be found at:

https://github.com/ALightTechnologyAndServicesLimited/LightKeysTransfer

var client = new SecretClient(new Uri("https://VAULT_NAME.vault.azure.net"), new DefaultAzureCredential());

client.SetSecret(new KeyVaultSecret(key, secret));

I don’t have any fake aliases, nor any virtual aliases like some of the the psycho spy R&AW traitors of India. NOT associated – “ass”, eass, female “es”, “eka”, “ok”, “okay”, “is”, “erra”, yerra, karan, kamalakar, diwakar, kareem, karan, sowmya, zinnabathuni, bojja srinivas (was a friend and batchmate 1998 – 2002), mukesh golla (was a friend and classmate 1998 – 2002), thota veera, uttam’s, bandhavi’s, bhattaru’s, thota’s, bojja’s, bhattaru’s or Arumilli srinivas or Arumilli uttam (may be they are part of a different Arumilli family – not my family).

Mr. Kanti Kalyan Arumilli

Arumilli Kanti Kalyan, Founder & CEO
Arumilli Kanti Kalyan, Founder & CEO

B.Tech, M.B.A

Facebook

LinkedIn

Threads

Instagram

Youtube

Founder & CEO, Lead Full-Stack .Net developer

ALight Technology And Services Limited

ALight Technologies USA Inc

Youtube

Facebook

LinkedIn

Phone / SMS / WhatsApp on the following 3 numbers:

+91-789-362-6688, +1-480-347-6849, +44-07718-273-964

+44-33-3303-1284 (Preferred number if calling from U.K, No WhatsApp)

kantikalyan@gmail.com, kantikalyan@outlook.com, admin@alightservices.com, kantikalyan.arumilli@alightservices.com, KArumilli2020@student.hult.edu, KantiKArumilli@outlook.com and 3 more rarely used email addresses – hardly once or twice a year.

Categories
.Net C#

Dapper for data access in C# part 2

In the past, I have written about A quick introduction to Dapper! This blog article is a continuation:

The most popular ways to access data in .Net are:

  1. ADO.Net
  2. Entity Framework
  3. 3rd party ORMs such as Dapper

ADO.Net needs a lot of code i.e opening IDataReader, iterating, reading elements and mapping etc… Similarly when calling Stored Procedures, adding params etc…

EntityFramework is a great choice for a lot of different situations.

When connecting to RDBMS such as SQL Server, MySQL, I prefer StoredProcs over dynamically generated SQL for several reasons. Consider this as part of some best practices from security and performance reasons rather than ease of code. In the past, I have been responsible and represented audits from development team perspective.

The 3rd choice – 3rd party ORMs and Micro ORMs. Dapper is a micro ORM. Dapper has excellent performance and probably the most suitable when handling Stored Procedures. Dapper supports many different RDBMS types.

Even Dapper call’s can be easily wrapped around and logging etc… can be done. Almost all the methods accept optional Transaction for wrapping multiple calls in a single transaction.

https://www.learndapper.com/ has excellent documentation.

Some of the nicer features of Dapper are:

  1. No need of strongly typed objects
  2. Multiple RDBMS support
  3. Ability to read multiple resultsets

Some code samples (MySQL example):

using Dapper;

using (var connection = new MySqlConnection(connectionString))
{
    await connection.OpenAsync();
    await connection.ExecuteAsync(
       "STORED_PROC_NAME", 
       new {
          varParam1 = "value1",
          varParam2 = 1
       }, null, null, CommandType.StoredProcedure);
}

Some of the other useful methods are:

connection.ExecuteReaderAsync //For any reason, if you need IDataReader

ExecuteScalar // For single object
            // or
ExecuteScalar<T> //For single object of type T
            // or
ExecuteScalarAsync(), ExecuteScalaAsync<T>() // The async equivalents

Query()
QueryFirst()
QueryFirstOrDeafult()
QuerySingle()

 

BTW, combining these with Polly and creating robust wrappers would be a recommended approach.

Polly library for writing resilient .Net code

I don’t have any fake aliases, nor any virtual aliases like some of the the psycho spy R&AW traitors of India. NOT associated – “ass”, eass, female “es”, “eka”, “ok”, “okay”, “is”, “erra”, yerra, karan, kamalakar, diwakar, kareem, karan, sowmya, zinnabathuni, bojja srinivas (was a friend and batchmate 1998 – 2002), mukesh golla (was a friend and classmate 1998 – 2002), thota veera, uttam’s, bandhavi’s, bhattaru’s, thota’s, bojja’s, bhattaru’s or Arumilli srinivas or Arumilli uttam (may be they are part of a different Arumilli family – not my family).

Mr. Kanti Kalyan Arumilli

Arumilli Kanti Kalyan, Founder & CEO
Arumilli Kanti Kalyan, Founder & CEO

B.Tech, M.B.A

Facebook

LinkedIn

Threads

Instagram

Youtube

Founder & CEO, Lead Full-Stack .Net developer

ALight Technology And Services Limited

ALight Technologies USA Inc

Youtube

Facebook

LinkedIn

Phone / SMS / WhatsApp on the following 3 numbers:

+91-789-362-6688, +1-480-347-6849, +44-07718-273-964

+44-33-3303-1284 (Preferred number if calling from U.K, No WhatsApp)

kantikalyan@gmail.com, kantikalyan@outlook.com, admin@alightservices.com, kantikalyan.arumilli@alightservices.com, KArumilli2020@student.hult.edu, KantiKArumilli@outlook.com and 3 more rarely used email addresses – hardly once or twice a year.

Categories
.Net C# Logging NLog

Custom Layout Renderer for NLog using C# New

In a previous blog post – Custom Layout Renderer for NLog using C#, I have mentioned about how to create a class inheriting LayoutRenderer and writing custom Layout Renderer.

This blog post talks about inheriting from WrapperLayoutRendererBase class.

The WrapperLayoutRendererBase class has a method with the signature:

protected override string Transform(string text)

This method needs to be overridden. The nice thing about using this class is other layout renderers can be used in combination.

Here is a GitHub repo implementing hash functions that can be used as LayoutRenderers.

https://github.com/ALightTechnologyAndServicesLimited/ALight.NLog.LayoutRenderer.Hash

The source code has 4 projects:

  1. The actual implementation
  2. Unit Tests
  3. Benchmark Tests
  4. Example

The usage is straightforward, use hash or securehash layout renderer.

  <extensions>
    <add assembly="ALight.NLog.LayoutRenderer.Hash" />
  </extensions>




<target xsi:type="File" name="logfile" fileName="c:\temp\console-example.log"
        layout="${longdate}|${level}|${message} |${all-event-properties} ${exception:format=tostring} ${hash:hello} 
        ${securehash:value=${level}} ${event-properties:item=secret} ${hash:${event-properties:item=secret}} ${hash:${level}}" />

The hash implementation is about 10 – 11 times faster but uses Murmur3 non cryptographic hash.

I might consider a nuget package at a later point, but not now, because the code is very minimal and like a small helper class.

I don’t have any fake aliases, nor any virtual aliases like some of the the psycho spy R&AW traitors of India. NOT associated – “ass”, eass, female “es”, “eka”, “ok”, “okay”, “is”, “erra”, yerra, karan, kamalakar, diwakar, kareem, karan, sowmya, zinnabathuni, bojja srinivas (was a friend and batchmate 1998 – 2002), mukesh golla (was a friend and classmate 1998 – 2002), thota veera, uttam’s, bandhavi’s, bhattaru’s, thota’s, bojja’s, bhattaru’s or Arumilli srinivas or Arumilli uttam (may be they are part of a different Arumilli family – not my family).

Mr. Kanti Kalyan Arumilli

Arumilli Kanti Kalyan, Founder & CEO
Arumilli Kanti Kalyan, Founder & CEO

B.Tech, M.B.A

Facebook

LinkedIn

Threads

Instagram

Youtube

Founder & CEO, Lead Full-Stack .Net developer

ALight Technology And Services Limited

ALight Technologies USA Inc

Youtube

Facebook

LinkedIn

Phone / SMS / WhatsApp on the following 3 numbers:

+91-789-362-6688, +1-480-347-6849, +44-07718-273-964

+44-33-3303-1284 (Preferred number if calling from U.K, No WhatsApp)

kantikalyan@gmail.com, kantikalyan@outlook.com, admin@alightservices.com, kantikalyan.arumilli@alightservices.com, KArumilli2020@student.hult.edu, KantiKArumilli@outlook.com and 3 more rarely used email addresses – hardly once or twice a year.

Categories
.Net C# Logging NLog

Custom Layout Renderer for NLog using C#

*** use WrapperLayoutRendererBase instead of LayoutRenderer, this blog post is about LayoutRenderer ***

NLog has lot of built-in layout renderers. Here is a list of official layout renderers:

https://nlog-project.org/config/?tab=layout-renderers

Creating custom layout renderer is very simple and straight-forward. Here is the official documentation:

https://github.com/NLog/readthedocs/blob/master/docs/How-to-write-a-custom-layout-renderer.md

There are 2 ways:

Lambda:

LayoutRenderer.Register("name", (logEvent, config) => "Message");

In the above scenarion, we can use ${name} when defining a layout, of course instead of returning static string we can do something useful based on the use-case.

Class:

Define a new class inherited from NLog.LayoutRenderers.LayoutRenderer. Use class attribute LayoutRenderer(“name”). LayoutRenderer is an abstract class, implement the class by overriding Append(StringBuilder builder, LogEventInfo logEvent) method i.e Append the value that needs to be added to the log by appending to the StringBuilder parameter.

[LayoutRenderer("Example")]
public class ExampleLayoutRenderer : NLog.LayoutRenderers.LayoutRenderer
{
    public string Value { get; set; }

    protected override void Append(StringBuilder builder, LogEventInfo logEvent)
    {
        builder.Append("Value");
    }
}

Extra parameters can be passed and assigned as properties of the custom class.

[LayoutRenderer("example")]
public class ExampleLayoutRenderer : NLog.LayoutRenderers.LayoutRenderer
{
    public string Value { get; set; }

    protected override void Append(StringBuilder builder, LogEventInfo logEvent)
    {
        // Your custom code
        builder.Append(Value);
    }
}

In the config file the custom layout renderer with params can be used like follows:

{example}

{example:value=abc}

Sometimes, in logs we write some data that might be needed for identifying but not necessarily the real value and for various reasons we might just need a hash. For example: SessionId / Some Cookie Value / IP address etc… Having a custom renderer for hashing such a value could be used.

Very few lines of code, i.e hashing and appending to a stringbuilder would solve the need i.e 2 – 3 lines of code. But, I think the 2 – 3 lines of code could be helpful for other people. I am considering creating an opensource version. I might or might not release nuget package. If I release nuget package, I would definitely make an anouncement. Anyway, not a big open-source project, but just few lines of code.

I don’t have any fake aliases, nor any virtual aliases like some of the the psycho spy R&AW traitors of India. NOT associated – “ass”, eass, female “es”, “eka”, “ok”, “okay”, “is”, “erra”, yerra, karan, kamalakar, diwakar, kareem, karan, sowmya, zinnabathuni, bojja srinivas (was a friend and batchmate 1998 – 2002), mukesh golla (was a friend and classmate 1998 – 2002), thota veera, uttam’s, bandhavi’s, bhattaru’s, thota’s, bojja’s, bhattaru’s or Arumilli srinivas or Arumilli uttam (may be they are part of a different Arumilli family – not my family).

Mr. Kanti Kalyan Arumilli

Arumilli Kanti Kalyan, Founder & CEO
Arumilli Kanti Kalyan, Founder & CEO

B.Tech, M.B.A

Facebook

LinkedIn

Threads

Instagram

Youtube

Founder & CEO, Lead Full-Stack .Net developer

ALight Technology And Services Limited

ALight Technologies USA Inc

Youtube

Facebook

LinkedIn

Phone / SMS / WhatsApp on the following 3 numbers:

+91-789-362-6688, +1-480-347-6849, +44-07718-273-964

+44-33-3303-1284 (Preferred number if calling from U.K, No WhatsApp)

kantikalyan@gmail.com, kantikalyan@outlook.com, admin@alightservices.com, kantikalyan.arumilli@alightservices.com, KArumilli2020@student.hult.edu, KantiKArumilli@outlook.com and 3 more rarely used email addresses – hardly once or twice a year.

Categories
.Net C# Cryptography

BenchmarkDotNet with hash implementation in C#

In earlier blog post:

I have discussed about cryptographic and non-cryptographic hashes.

Here is a Github repo associated with this:

https://github.com/ALightTechnologyAndServicesLimited/Benchmark-Hash-Implementations-in-CSharp

For using BenchmarkDotNet:

Add the nuget package to a console application:

dotnet add package BenchmarkDotNet --version 0.13.7

Create a class and use the attributes [GlobalSetup] for any initialization, use the [Benchmark] attribute for the methods. Use [MemoryDiagnoser] for memory related information.

In the Program.cs use:

BenchmarkRunner.Run<BENCHMARK_CLASS>();

The code in the github repo clearly demonstrates the usage.

In the previous blog post I have mentioned about non-cryptographic hashes for general purposes.

Here are the results:

32 bytes data:

I commented some code and evaluated with more data sizes:

25 KB:

5 MB:

25 MB:

Conclusion:

For non-cryptographic 128 bit hashes use Murmur3.

I don’t have any fake aliases, nor any virtual aliases like some of the the psycho spy R&AW traitors of India. NOT associated – “ass”, eass, female “es”, “eka”, “ok”, “okay”, “is”, “erra”, yerra, karan, kamalakar, diwakar, kareem, karan, sowmya, zinnabathuni, bojja srinivas (was a friend and batchmate 1998 – 2002), mukesh golla (was a friend and classmate 1998 – 2002), thota veera, uttam’s, bandhavi’s, bhattaru’s, thota’s, bojja’s, bhattaru’s or Arumilli srinivas or Arumilli uttam (may be they are part of a different Arumilli family – not my family).

Mr. Kanti Kalyan Arumilli

Arumilli Kanti Kalyan, Founder & CEO
Arumilli Kanti Kalyan, Founder & CEO

B.Tech, M.B.A

Facebook

LinkedIn

Threads

Instagram

Youtube

Founder & CEO, Lead Full-Stack .Net developer

ALight Technology And Services Limited

ALight Technologies USA Inc

Youtube

Facebook

LinkedIn

Phone / SMS / WhatsApp on the following 3 numbers:

+91-789-362-6688, +1-480-347-6849, +44-07718-273-964

+44-33-3303-1284 (Preferred number if calling from U.K, No WhatsApp)

kantikalyan@gmail.com, kantikalyan@outlook.com, admin@alightservices.com, kantikalyan.arumilli@alightservices.com, KArumilli2020@student.hult.edu, KantiKArumilli@outlook.com and 3 more rarely used email addresses – hardly once or twice a year.

Categories
.Net C# Cryptography

A discussion of cryptographic hash and non-cryptographic hash implementations using C#

Hash algorithms take some data and create fixed size representation. The representations can be of various sizes such as 32/64/128/256/512 bytes.

Hash generated are in-general non-reversible i.e you can’t get the original data from hash.

Cryptographic hashes use some high level of cryptography and should be used for sensitive information such as passwords etc…

Non-cryptographic hash implementations can be used for general information.

SHA256, SHA384 and SHA512 are for cryptographic hashes.

The above mentioned SHA implementations are secure but little slower.

Here are some faster hash implementations for non-secure data purposes:

FastHash – Can generate 32/64/128 bit hashes.

var sometext = "SOME TEXT";

ReadOnlySpan<byte> data = new ReadOnlySpan<byte>(Encoding.UTF8.GetBytes(sometext));

var retVal = FastHash.HashGenerator.GenerateHash128(data);

var guid = retVal.AsGuid().ToString();
          OR
Convert.ToBase64String(retVal.AsSpan());

MurmurHash3 – Generates 128 bit hash and faster than FastHash’s 128 bit implementation.

var sometext = "SOME TEXT";

var data = Encoding.UTF8.GetBytes(sometext);

var retVal = murmurHasher.ComputeHash(data);

var guid = Convert.ToBase64String(retVal);

BLAKE2 – Generates 1-64 bytes i.e 8-512 bits hash.

var sometext = "SOME TEXT";

var bytes2 = Blake2Fast.Blake2b.ComputeHash(new ReadOnlySpan<byte>(Encoding.UTF8.GetBytes(sometext)));

var guid = Convert.ToBase64String(bytes2);

For specifying the hash-size in bytes use the overload for ComputeHash.

var bytes2 = Blake2Fast.Blake2b.ComputeHash(new ReadOnlySpan<byte>(<bytes>, Encoding.UTF8.GetBytes(sometext)));

The value of <bytes> can be between 1 and 64.

I don’t have any fake aliases, nor any virtual aliases like some of the the psycho spy R&AW traitors of India. NOT associated – “ass”, eass, female “es”, “eka”, “ok”, “okay”, “is”, “erra”, yerra, karan, kamalakar, diwakar, kareem, karan, sowmya, zinnabathuni, bojja srinivas (was a friend and batchmate 1998 – 2002), mukesh golla (was a friend and classmate 1998 – 2002), thota veera, uttam’s, bandhavi’s, bhattaru’s, thota’s, bojja’s, bhattaru’s or Arumilli srinivas or Arumilli uttam (may be they are part of a different Arumilli family – not my family).

Mr. Kanti Kalyan Arumilli

Arumilli Kanti Kalyan, Founder & CEO
Arumilli Kanti Kalyan, Founder & CEO

B.Tech, M.B.A

Facebook

LinkedIn

Threads

Instagram

Youtube

Founder & CEO, Lead Full-Stack .Net developer

ALight Technology And Services Limited

ALight Technologies USA Inc

Youtube

Facebook

LinkedIn

Phone / SMS / WhatsApp on the following 3 numbers:

+91-789-362-6688, +1-480-347-6849, +44-07718-273-964

+44-33-3303-1284 (Preferred number if calling from U.K, No WhatsApp)

kantikalyan@gmail.com, kantikalyan@outlook.com, admin@alightservices.com, kantikalyan.arumilli@alightservices.com, KArumilli2020@student.hult.edu, KantiKArumilli@outlook.com and 3 more rarely used email addresses – hardly once or twice a year.

Categories
.Net C# NLog UnitTests

Code Coverage in .Net 2023

In the past, I have written a blog post about code coverage using NUnit, OpenCover and ReportGenerator https://www.alightservices.com/2022/06/23/code-coverage-in-net/.

OpenCover development has stopped for over 2 years.

This blog post talks about another tool known as Coverlet.

This blog posts has a accompanying Github repo – https://github.com/ALightTechnologyAndServicesLimited/CodeCoverage/

The code has 4 methods regarding DateTime class. And 3 helper methods:

  • double GetUnixEpoch(DateTime)
  • DateTime GetDateTimeFromUnixEpoch(double)
  • long GetYYYMMDD(DateTime)
  • bool IsLeapYear(int)

There are 9 unit tests for the above mentioned 4 methods.

The batch file – RunCodeCoverage.bat has the commands.

The usage is very simple, add the Coverlet nuget package, install the ReportGenerator tool and run some commands. Navigate to the unittest project.

dotnet add package coverlet.collector

dotnet tool install -g dotnet-reportgenerator-globaltool

The nuget package coverlet.collector needs to be added for every unit test project. The reportgenerator can be installed once.

Then run this command for invoking coverlet:

dotnet test --collect:"XPlat Code Coverage"

This generates a xml file under TestResults/[some-random-guid].

This xml file would be used by reportgenerator for generating HTML report:

reportgenerator.exe "-reports:TestResults**.*.xml" "-targetdir:report"

The above command looks for xml files under TestResults/* folder and generates HTML reports under report folder.

The HTML report looks like this. If there is any code that was not covered under unit test, those can be seen easily. The following screenshots have 100% code coverage, but if there is any code not covered, the code would be shown in red.

I don’t have any fake aliases, nor any virtual aliases like some of the the psycho spy R&AW traitors of India. NOT associated – “ass”, eass, female “es”, “eka”, “ok”, “okay”, “is”, “erra”, yerra, karan, kamalakar, diwakar, kareem, karan, sowmya, zinnabathuni, bojja srinivas (was a friend and batchmate 1998 – 2002), mukesh golla (was a friend and classmate 1998 – 2002), thota veera, uttam’s, bandhavi’s, bhattaru’s, thota’s, bojja’s, bhattaru’s or Arumilli srinivas or Arumilli uttam (may be they are part of a different Arumilli family – not my family).

Mr. Kanti Kalyan Arumilli

Arumilli Kanti Kalyan, Founder & CEO
Arumilli Kanti Kalyan, Founder & CEO

B.Tech, M.B.A

Facebook

LinkedIn

Threads

Instagram

Youtube

Founder & CEO, Lead Full-Stack .Net developer

ALight Technology And Services Limited

ALight Technologies USA Inc

Youtube

Facebook

LinkedIn

Phone / SMS / WhatsApp on the following 3 numbers:

+91-789-362-6688, +1-480-347-6849, +44-07718-273-964

+44-33-3303-1284 (Preferred number if calling from U.K, No WhatsApp)

kantikalyan@gmail.com, kantikalyan@outlook.com, admin@alightservices.com, kantikalyan.arumilli@alightservices.com, KArumilli2020@student.hult.edu, KantiKArumilli@outlook.com and 3 more rarely used email addresses – hardly once or twice a year.