Add free search for your website. Sign up now! https://webveta.alightservices.com/
Categories
Welcome

Some helpful database commands in MySQL / MariaDB

I personally am NOT a DBA. But as a one person company I do everything from product planning, feature prioritization, architecture, documentation, development, deployment, DBA and monitoring. Below are some helpful commands in MySQL / MariaDB that I found useful when I do DBA activities. This is more of a blog post for my own reference.

Connect to database:

mysql -u root -p

Get a list of databases:

show databases;

Use a particular database

use <DATABASE_NAME>;

Show tables in current database

show tables;

Get a list of all the stored procedures in all the databases

SHOW PROCEDURE STATUS;

or

SELECT 
    routine_schema as "Database",
    routine_name
FROM 
    information_schema.routines
WHERE 
    routine_type = 'PROCEDURE'
ORDER BY 
    routine_schema ASC, 
    routine_name ASC;

Get a list of stored procedures in all the databases that match a certain string pattern.

SHOW PROCEDURE STATUS LIKE '%pattern%';

or

SELECT 
    routine_schema as "Database",
    routine_name
FROM 
    information_schema.routines
WHERE 
    routine_type = 'PROCEDURE'
    and routine_name LIKE '%pattern%'
ORDER BY 
    routine_schema ASC, 
    routine_name ASC;

The second select statement can be modified to limit to a certain database by including filters in the where clause on “routine_schema”.

SELECT 
    routine_schema as "Database",
    routine_name
FROM 
    information_schema.routines
WHERE 
    routine_type = 'PROCEDURE'
    and routine_name LIKE '%pattern%'
    and routine_schema = 'database_name'
ORDER BY 
    routine_schema ASC, 
    routine_name ASC;

Hoping this helps someone!

Never trust the scammers who claim to have exchanged identities and who use bank accounts on other people’s name. They are either identity thieves / impersonators / anonymous hackers. Example – Erra’s, Thota’s, Bojja’s, Uttam’s, Srinivas’s etc… the R&AW hackers / spies.

Mr. Kanti Kalyan Arumilli

Arumilli Kanti Kalyan, Founder & CEO
Arumilli Kanti Kalyan, Founder & CEO

B.Tech, M.B.A

Facebook

LinkedIn

Threads

Instagram

Youtube

Founder & CEO, Lead Full-Stack .Net developer

ALight Technology And Services Limited

ALight Technologies USA Inc

Youtube

Facebook

LinkedIn

Phone / SMS / WhatsApp on the following 3 numbers:

+91-789-362-6688, +1-480-347-6849, +44-07718-273-964

kantikalyan@gmail.com, kantikalyan@outlook.com, admin@alightservices.com, kantikalyan.arumilli@alightservices.com, KArumilli2020@student.hult.edu, KantiKArumilli@outlook.com and 3 more rarely used email addresses – hardly once or twice a year.

Categories
.Net C# gRPC

Advantages of gRPC vs REST

REST – Representational State Transfer is the most common way of data communication and is based on HTTP 1.1. The data format are generally XML or JSON. HTTP Status codes are usually used for statuses, and sometimes the payload has the statuses.

gRPC – Google’s Remote Procedure Call is a more modern method and based on HTTP 2. Due to HTTP 2 some older software or browsers might not have support. But, gRPC is the way forward. The advantages are several, I am mentioning some of the advantages here:

  1. High performance serializer and deserializer based on protobuf binary format.
  2. The data size is much lesser compared with JSON / XML.
  3. Server to client communication (based on existing connection), Client to server communication, Streaming from Server to Client (on existing connection), Streaming from Client to Server and even duplex communication.
  4. Efficient usage of networking i.e because gRPC is based on HTTP 2, network connections need not be opened for every call.

.Net has fully embraced and supports modern code generation for gRPC. In further blog posts, I will explain and provide some code samples for using gRPC in .Net.

I personally have not performed any speed tests trying to compare REST vs gRPC but, I did use gRPC in some micro-service architecture application and found the performance of gRPC significantly higher than REST.

Mr. Kanti Kalyan Arumilli

Arumilli Kanti Kalyan, Founder & CEO
Arumilli Kanti Kalyan, Founder & CEO

B.Tech, M.B.A

Facebook

LinkedIn

Threads

Instagram

Youtube

Founder & CEO, Lead Full-Stack .Net developer

ALight Technology And Services Limited

ALight Technologies USA Inc

Youtube

Facebook

LinkedIn

Phone / SMS / WhatsApp on the following 3 numbers:

+91-789-362-6688, +1-480-347-6849, +44-07718-273-964

kantikalyan@gmail.com, kantikalyan@outlook.com, admin@alightservices.com, kantikalyan.arumilli@alightservices.com, KArumilli2020@student.hult.edu, KantiKArumilli@outlook.com and 3 more rarely used email addresses – hardly once or twice a year.

Categories
.Net ASP.Net C#

Dependency Injection in ASP.Net MVC Core explained

Dependency Injection is a software development pattern where instead of directly instantiating objects, the objects required by a class are passed in. This helps with maintaining code flexibility, writing unit test cases etc…

The first and foremost thing is to define interfaces and then write implementations. This way, the consuming code needs to know about the methods to be invoked without worrying about the implementation. Software known as Dependency Injection container takes care of instantiating the actual objects as long as the bindings are defined.

This blog post is not about Dependency Injection or Unit Tests but more about how to use Dependency Injection in ASP.Net MVC Core. ASP.Net MVC Core comes with an in-built DI container and supports constructor-based injection i.e instances are passed into the constructor of the consuming class.

There are 3 scopes for objects:

Transient: Every time a class needs an object, a new instance of the requested object is instantiated and passed in. i.e for example if there are 3 classes that need an instance of IService, each class will receive it’s own copy every time even if the three classes are used as part of the same request/response.

Scoped: One object for a particular type is created per request/response and the same object is passed into every class that requests the object processing one request/response cycle.

Singleton: One instance of the class is instantiated for the entire lifetime of the application and the same instance is passed for every class in every request/response cycle.

The use cases for each would vary. Scoped is the default i.e one object for a given type for every class in the same request/response cycle.

Singleton’s are useful in cases such as IConfiguration where the same class can be passed around for getting config information rather than having multiple instances.

Interfaces and implementation classes can be registered by calling the following methods on IServiceCollection for example

AddSingleton<IInterface, Implementation>();
AddScoped<IInterface, Implementation>();
AddTransient<IInterface, Implementation>();

or in Singleton if the object is already instantiated, the object can be passed in by calling:

AddSingleton<IInterface>(instance);

Mr. Kanti Kalyan Arumilli

Arumilli Kanti Kalyan, Founder & CEO
Arumilli Kanti Kalyan, Founder & CEO

B.Tech, M.B.A

Facebook

LinkedIn

Threads

Instagram

Youtube

Founder & CEO, Lead Full-Stack .Net developer

ALight Technology And Services Limited

ALight Technologies USA Inc

Youtube

Facebook

LinkedIn

Phone / SMS / WhatsApp on the following 3 numbers:

+91-789-362-6688, +1-480-347-6849, +44-07718-273-964

kantikalyan@gmail.com, kantikalyan@outlook.com, admin@alightservices.com, kantikalyan.arumilli@alightservices.com, KArumilli2020@student.hult.edu, KantiKArumilli@outlook.com and 3 more rarely used email addresses – hardly once or twice a year.

Categories
.Net ASP.Net C#

ViewComponents in ASP.Net and caching ViewComponents

ViewComponent’s are pretty much like PartialView’s but slightly more useful. ViewComponent’s help in rendering part’s of a web page that can be re-used across the website. Also ViewComponent’s output can be cached. This blog article is going to discuss creating ViewComponent’s and caching example.

A ViewComponent is a public class that has a ViewComponent suffix such as HeaderViewComponent or MenuViewComponent etc… ViewComponent class can be decorated with [ViewComponent] attribute or can inherit from ViewComponent class or any other class that’s a ViewComponent. For example some kind of a BaseViewComponent.

ViewComponent must have one method that gets called.

async Task<IViewComponentResult> InvokeAsync()
or
IViewComponentResult Invoke()

The runtime by default searches for the Views in the following locations:

The runtime searches for the view in the following paths:

  • /Views/{Controller Name}/Components/{View Component Name}/{View Name}
  • /Views/Shared/Components/{View Component Name}/{View Name}
  • /Pages/Shared/Components/{View Component Name}/{View Name}

ViewComponent gets invoked from cshtml by using: Component.InvokeAsync()

The call to Component.InvokeAsync() can be wrapped inside <cache> tag helper for caching.

With the concepts discussed above, let’s look at a code sample. Assuming you have a ASP.Net MVC Core test project opened. Now add a new class and name the class TestViewComponent in TestViewComponent.cs.

using Microsoft.AspNetCore.Mvc;


namespace TestProject.ViewComponents
{
    public class TestViewComponent : ViewComponent
    {
        public async Task<IViewComponentResult> InvokeAsync()
        {
            return await Task.FromResult((IViewComponentResult)View("Test"));
        }
    }
}

Now under Views/Shared create a folder and name the folder Components. Under Views/Shared/Components, create another folder Test. Now, Views/Shared/Components/Test folder can contain views for the TestViewComponent. Create a new Test.cshtml under Views/Shared/Components/Test and put some random html content.

<p>Hello from TestViewComponent.</p>

Now somewhere on Views/Home/Index.cshtml place the following invocation:

@(await Component.InvokeAsync("Test"))

If you need to cache the output wrap the invocation inside <cache> tag helper.

        <cache expires-after="@TimeSpan.FromMinutes(5)">
            @(await Component.InvokeAsync("Test"))
        </cache>

Hope this blog post helps someone!

Mr. Kanti Kalyan Arumilli

Arumilli Kanti Kalyan, Founder & CEO
Arumilli Kanti Kalyan, Founder & CEO

B.Tech, M.B.A

Facebook

LinkedIn

Threads

Instagram

Youtube

Founder & CEO, Lead Full-Stack .Net developer

ALight Technology And Services Limited

ALight Technologies USA Inc

Youtube

Facebook

LinkedIn

Phone / SMS / WhatsApp on the following 3 numbers:

+91-789-362-6688, +1-480-347-6849, +44-07718-273-964

kantikalyan@gmail.com, kantikalyan@outlook.com, admin@alightservices.com, kantikalyan.arumilli@alightservices.com, KArumilli2020@student.hult.edu, KantiKArumilli@outlook.com and 3 more rarely used email addresses – hardly once or twice a year.

Categories
.Net C# Solr

Some performance tips when ingesting documents into Solr

As mentioned in several blog posts earlier, I have been building PodDB on Microsoft.Net platform and Solr. Solr is built on top of Apache Lucene.

Lucene.Net is a very high-performance library for working directly with Apache Lucene, SolrNet is a library for working with Solr. Solr is very customizable, fault-tolerant and has several additional features available out of the box and is built on top of Lucene. Working with SolrNet can be a bit slow because all the API calls are routed via a REST API. The usual overhead of establishing network connection, serializing and deserializing JSON or XML.

Over the past few days, I have been working on a small subset of documents (approximately 275 – 300, the same would be part of the Alpha release) and trying to tweak the settings for optimal search relevance. This required trying various Solr configurations, re-indexing data etc… The very first version of the data ingestion component (does much more pre-processing rather than just ingesting into solr) used to take about approximately 10 minutes. And now the performance has been optimized and the ingestion happens within 15 seconds. i.e over 4000% performance gain and entirely programming related.

The trick used was one of the oldest tricks in the book – batch processing. Instead of one document at a time for writing into a MySQL database and writing into Solr, I rewrote the application to ingest in batches and the application was much faster.

Batching with multi-threading might be even faster.

In other words instead of calling solr.Add() for each document, create the documents, hold them in a list, call solr.AddRange().

Similarly for solr.Commit() and solr.Optimize() batch the calls i.e call those methods once for every 1000 or so documents rather than every document.

Assuming doc is a Solr document that needs to be written. For example:

//NO
solr.Add(doc1);
solr.Add(doc2);
solr.Add(doc3);

//YES
var lst = new List<ENTITY>();
lst.Add(doc1);
lst.Add(doc2);
lst.Add(doc3);

solr.AddRange(lst);

I like to share knowledge, I am hoping this blog post helps someone.

My 2 cents to the world of the blogosphere!

Mr. Kanti Kalyan Arumilli

Arumilli Kanti Kalyan, Founder & CEO
Arumilli Kanti Kalyan, Founder & CEO

B.Tech, M.B.A

Facebook

LinkedIn

Threads

Instagram

Youtube

Founder & CEO, Lead Full-Stack .Net developer

ALight Technology And Services Limited

ALight Technologies USA Inc

Youtube

Facebook

LinkedIn

Phone / SMS / WhatsApp on the following 3 numbers:

+91-789-362-6688, +1-480-347-6849, +44-07718-273-964

kantikalyan@gmail.com, kantikalyan@outlook.com, admin@alightservices.com, kantikalyan.arumilli@alightservices.com, KArumilli2020@student.hult.edu, KantiKArumilli@outlook.com and 3 more rarely used email addresses – hardly once or twice a year.

Categories
MySQL

Connecting to MySQL or MariaDB from Linux Command Line

There could be several reasons for working with MySQL from command prompt such as via SSH etc… This is a brief tutorial on some important mysql commands.

Connect by issuing the following command:

> mysql -u root -p

Then you might be prompted for password and after entering the correct password, MySQL shell would be accessible.

The following are some important commands for viewing databases, creating databases, creating users, and providing privileges. The commands are pretty much self-explanatory:

> show databases; -- Shows databases

> use <DATABASE_NAME>; -- Uses database, i.e next set of commands would be against the database, can be used on a necessity basis

> show tables; -- Viewing existing tables in a particular schema

> CREATE DATABSE <DATABASE_NAME>; - Creates a new schema

> CREATE USER 'new_user'@'localhost' IDENTIFIED BY 'password'; -- Creates a new user with userid of "new_user" and password of "password"

> GRANT PRIVILEGE ON database.table TO 'username'@'localhost'; -- Grants privileges on the specific table in the specific database

> GRANT PRIVILEGE ON database.* TO 'username'@'localhost'; -- Grants privileges on all tables in the specific database

> FLUSH PRIVILEGES; -- Not necessary but for the purpose of completeness

> exit; -- Exiting MySQL shell

MariaDB also has the same syntax.

Hoping this blog post helps someone.

Categories
.Net C#

A quick introduction to Dapper!

Dapper is a micro ORM tool with very high performance and very decent features. Dapper is one of my favourite tools. Dapper has excellent documentation here and here.

Dapper supports SQL statements and Stored Procedures. My preference is usually Stored Procs over SQL statements.

Dapper extends the IDbConnection interface and thus several methods are added on the connection object.

If you have a class Person with Id and Name, Dapper can handle mapping:

using(var connection = new MySqlConnection(connectionString){
    await connection.OpenAsync();

    var persons = await connection.QueryAsync<Person>("SELECT Id, Name FROM Person WHERE ....");
}

The above code snippet shows how to query the database and get a IEnumerable of Person objects.

There are several other methods, and each method has both synchronous and asynchronous versions, some of them have Generic versions for mapping to objects such as:

.Execute / ExecuteAsync
.ExecuteReader / ExecuteReaderAsync
.ExecuteScalar / ExecuteScalarAsync
.ExecuteScalar<T> / ExecuteScalarAsync<T>
etc...

Passing in parameters is also very straightforward, parameters can be passed in as an existing object or

new {Id = 1}

Even transactions and list of objects are supported.

Multiple resultsets are supported etc… If you are familiar with ADO.Net, using Dapper would be very easy, straightforward and much easier and has very excellent performance with minimal overhead.

Entity Framework is a close 2nd choice when dealing with Database-First approach. If using Code-First approach, Entity Framework would be the preferred choice.

Hoping this blog post helps someone.

Categories
Solr

Schemas and Configs in Solr

Solr has very extensive documentation and highly configurable. But for the purposes of getting started and keeping things simple, I am going to mention some important parts.

Solr can be operated with schema or schemaless. I personally would prefer proper schema. Solr allows storing values, skipping storage and just indexing or just storing without indexing. Similarly required or not, multi-valued, unique key etc…

Continuing from the previous blog posts of Getting started:

Getting started with Solr on Windows

Getting started with Solr on Linux!

First create a core by navigating to the solr directory and executing:

> solr.cmd create -c <NAME_OF_CORE>
// Example for creating a core known as "sample"
> solr.cmd create -c sample

Now navigate to <solr directory>\server\solr. Here you will find the sample directory. Inside sample directory, there would be a conf directory. The conf directory has the schema.xml and solrconfig.xml.

Now let’s delve into some Schema.xml:

The most important part is defining the schema, the copyField’s.

Let’s assume our schema requires a unique id, title, description and tags.

id is going to be a unique field, we want title, description and tags to be indexed. Same object can have multiple tags, so tags would be multi-valued.

We would add the following:

<field name="Title" type="text_general" indexed="true" stored="true"/>
<field name="Description" type="text_general" indexed="true" stored="true"/>
<field name="Tags" type="text_general" multiValued="true" indexed="true" stored="true"/>



<field name="id" type="string" multiValued="false" indexed="true" required="true" stored="true"/>

The “id” field would exist, we don’t want to add another “id” field, verify the syntax matches. Now we want to copy the values from Title, Description, Tags into _text_ and want _text_ to be indexed. Some people would try to copy * into _text_ as in all the fields. But in some schemas there may be meta information such as dates etc… So specifying which particular fields need to be copied would help. The next code block shows the copyfields.

<copyField source="Title" dest="_text_" />
<copyField source="Description" dest="_text_" />
<copyField source="Tags" dest="_text_" />

Now let’s delve into solrconfig.xml:

Search for <requestHandler name=”/select” class=”solr.SearchHandler”>

Inside this tag we can specify default page size, default field to index, any default facets etc. Let’s assume we want to set default page size of 10, default field to search of _text_ and faceting on Tags.

<requestHandler name="/select" class="solr.SearchHandler">
    <lst name="defaults">
      <str name="echoParams">explicit</str>
      <int name="rows">10</int>
      <str name="df">_text_</str>

      <str name="facet">on</str>
      <str name="facet.field">Tags</str>
    </lst>
  </requestHandler>

If config or schema changes, the core needs to be reloaded.

More Solr related Articles:

Schemas and Configs in Solr

Getting started with Solr on Windows

Getting started with Solr on Linux!

Hoping this blog post helps someone.

Categories
Solr

Getting started with Solr on Linux!

As promised in a previous blog post Getting started with Solr on Windows, posted few minutes ago, here is the Getting started on Linux.

First install Java. This blog post is based on Ubuntu 20.04 server.

> sudo apt update
> sudo apt upgrade
> sudo apt install default-jdk -y
> sudo wget https://dlcdn.apache.org/solr/solr/9.0.0/solr-9.0.0.tgz
> sudo tar xzf solr-9.0.0.tgz
> sudo bash solr-9.0.0/bin/install_solr_service.sh solr-9.0.0.tgz

Then navigate to http://localhost:8983/solr/ you should see a webpage that looks similar to:

Solr Admin Ubuntu

If you need access from a different computer, remember to allow port 8983 with the following command:

> sudo ufw allow 8983

Now we have seen Getting started with Solr on Windows and Getting started with Solr on Linux. In a future blog post, we will see how to create cores, define schema, add documents for indexing using C#, how to search through the documents using C#. This blog post would be updated with the links to the relevant blog articles when those blog posts are published.

More Solr related Articles:

Schemas and Configs in Solr

Getting started with Solr on Windows

Getting started with Solr on Linux!

Here are some excellent getting started documentation from Solr:

https://solr.apache.org/guide/solr/latest/getting-started/solr-tutorial.html

For production grade deployment:

https://solr.apache.org/guide/solr/latest/deployment-guide/solr-control-script-reference.html

https://solr.apache.org/guide/solr/latest/configuration-guide/configuration-files.html

Categories
Solr

Getting started with Solr on Windows

This blog post is purely about getting started with Solr. A programming related blog post would be posted later. Getting started on Linux would also be posted later.

Download the latest version of Solr from https://solr.apache.org/downloads.html. As of this blog post the latest release was 9.0.0 and I have downloaded the Binary release from https://www.apache.org/dyn/closer.lua/solr/solr/9.0.0/solr-9.0.0.tgz?action=download.

Once downloaded extract the files. For this blog post, I have extracted the files on Windows under C:\solr.

Solr requies Java. Download and install Java from https://www.oracle.com/java/technologies/downloads/. Set the environment variable JAVA_HOME to the JRE directory.

JAVA_HOME environment variable on Windows.

Now navigate to C:\Solr in command prompt and enter the following:

bin\solr.cmd start

This should start Solr, if not, verify your Java installation and environment variable.

Now navigate to http://localhost:8983/ in your web browser. You should see a web page that looks something like this:

Solr Admin Web Page

You might not see the Core Selector dropdown but the web page should look similar.

Getting started with Solr on Linux.

In a future blog post, we will see how to create cores, define schema, add documents for indexing using C#, how to search through the documents using C#. This blog post would be updated with the links to the relevant blog articles when those blog posts are published.

More Solr related Articles:

Schemas and Configs in Solr

Getting started with Solr on Windows

Getting started with Solr on Linux!

Here are some excellent getting started documentation from Solr:

https://solr.apache.org/guide/solr/latest/getting-started/solr-tutorial.html

For production grade deployment:

https://solr.apache.org/guide/solr/latest/deployment-guide/solr-control-script-reference.html

https://solr.apache.org/guide/solr/latest/configuration-guide/configuration-files.html