Add free search for your website. Sign up now! https://webveta.alightservices.com/

KickStarter campaign, offering steep discounts. Sign-up and support if possible. Thank you!

Month: September 2022

Categories
AWS

AWS CLI – Copying objects from and to S3

This is a very small blogpost about how to copy objects from S3 and to S3. The blog post assumes AWS CLI is installed and configured, and the user has appropriate permissions.

The command is simple and strightforward:

> aws s3 cp <src> <dest>

If copying from current computer to s3 an example would be:

> aws s3 cp "/path/file.zip" "s3://<BucketName>/Path/destinationFile.zip"

If copying from s3 to localhost the source and destination would be reversed.

> > aws s3 cp "s3://<BucketName>/Path/destinationFile.zip" "/path/file.zip"
Categories
.Net C# Lucene Solr

Lucene vs Solr

I played around with Lucene.Net and Solr. Solr is built on top of Lucene.

Lucene.Net is a port of Lucene library written in C# for working with Lucene on Microsoft .Net stack.

Lucene is a library built by Apache Software Foundation. Lucene provides full-text search capabilities. There are few other alternatives such as Sphinx, full-text search capabilities built into RDBMS’s such as Microsoft SQL Server, MySQL, MariaDB, PostgreSQL etc… However, full-text search capabilities in RDBMS’s are not as efficient as Lucene.

Solr and ElasticSearch are built on top of Lucene. ElasticSearch is more suitable and efficient for time-series data.

Now let’s see more about Solr vs Lucene.

Solr provides some additional features such as replication, web app GUI, collecting and publishing metrics, fault-tolerant etc… Solr provides HTTP REST-based API’s for management and for adding documents, searching documents etc…

Directly working with Lucene would provide access to more fine-grained control.

Because Solr provides REST based API’s there is the overhead of establishing HTTP connection, formatting the requests, JSON serialization, and deserialization at both ends i.e client making the call and the Solr server. By directly working with Lucene this overhead does not exist.

If searching through the documents happens on the same server, working directly with Lucene might be efficient. Specifically in lesser data scenarios, but if huge datasets and scaling are a concern, Solr might be the proper approach.

If server infrastructure requirements require separate search servers and a bunch of application servers query the search servers for data, Solr might be more useful and easier because of existing support replication and HTTP API’s.

If performance is of the highest importance and still fine-grained control is needed, custom-built applications should expose the data from search servers and some other more efficient protocols such as gRPC could be used and obviously, replication mechanisms need to be custom-built.

Categories
Github

How to access GitHub with SSH keys

Sometimes we need to access private repositories of a github account from linux servers for various purposes. For example, I have MFA enabled on my Github account, I need to clone a private repository inside a Linux server hosted in AWS. This article explains how to deal with such situations.

This is pretty much a re-hash of the steps provided in but in a slightly friendlier way.

Generating a new SSH key and adding it to the ssh-agent

Adding a new SSH key to your GitHub account

Login into your server and generate SSH keys by issuing the following command:

ssh-keygen -t ed25519 -C "<YOUR_EMAIL_ADDRESS>"

You would be prompted for a filename, password and re-type password. You can accept the defaults and click enter or enter a separate filename and password.

Now start the SSH agent by issuing the following command:

eval "$(ssh-agent -s)"

Now add the previously generated key for usage by issuing the following command:

ssh-add ~/.ssh/id_ed25519

Now either copy the content of the public key by using the clip command or view the content of the public key by using the cat command:

clip < ~/.ssh/id_ed25519.pub
or
cat < ~/.ssh/id_ed25519.pub

Now go to your GitHub account, sign-in and click on your profile on the right side top and click settings.

GitHub Profile Menu

From the left menu click SSH and GPG Keys

GitHub profile settings page’s left-side menu

Click New SSH Key

SSH Keys

Provide a title, a friendly name to remember, and enter the public key and click Add SSH key.

Add SSH key screen

Depending on your settings you might be prompted to re-authenticate or for MFA.

Now try cloning a repository from your Linux terminal. First copy the SSH URL of the repo:

Git clone URL screen

Then issue the following command on the Linux terminal to clone the main branch or a specific branch.

git clone <SSH_URL>
or
git clone --branch <BRANCH_NAME> <SSH_URL>

Hoping this helps someone :).