WebVeta - Advanced, unified, consistent search for your website(s), from content of your website(s), blogs(s). First 50 customers, who sign-up prior to 15/05/2024 get unlimited access to existing features, newer features for at least 1 year. Sign up now! https://webveta.alightservices.com/
Categories
C# Cloudwatch

Read Cloudwatch Logs Programatically using C#

In AWS, Cloudwatch is an extremely useful service for ingesting and retrieving logs, metrics, alarms etc… This particular blog post is about how to retrieve logs from cloudwatch.

Cloudwatch logs are organized in the following hierarchy: Region -> LogGroup -> Streams. Inside each stream are log messages which have 3 attributes: Message, Timestamp, Ingestion Time. Currently, I am using Cloudwatch for various logs such as Linux syslogs, web server logs, Cloudtrail events etc… How to programmatically ingest logs of .Net applications directly or using NLog or using Cloudwatch agent would be topics of future blog posts. This blog post assumes that the .Net code is running under an appropriate role that has appropriate permissions.

Install AWSSDK.CloudwatchLogs nuget package.

// Instantiating a Cloudwatch client
AmazonCloudWatchLogsClient client = new AmazonCloudWatchLogsClient(RegionEndpoint.EUWest2);


// Getting Log Groups - Code snippet
var logGroupsresponse = await client.DescribeLogGroupsAsync();

if(logGroupsresponse.HttpStatusCode == HttpStatusCode.OK)
{
   foreach(var logGroup in logGroupsresponse.LogGroups)
   {
         // Process
   }
}


// Getting Streams - Code snippet
var streamResponse = await client.DescribeLogStreamsAsync(
    new DescribeLogStreamsRequest
    {
        LogGroupName = "LogGroupName"
    });

if(streamResponse.HttpStatusCode == HttpStatusCode.OK)
{
   foreach(var stream in streamResponse.LogStreams)
   {
         // Process
   }
}


// Getting Log Messages - Code snippet
var logEventsresponse = await client.GetLogEventsAsync(
    new GetLogEventsRequest
    {
        LogGroupName = "Log Group Name",

        LogStreamName = "Log Stream Name"
    });

if(logEventsresponse.HttpStatusCode == HttpStatusCode.OK)
{
   foreach(var logMessage in logEventsresponse.Events)
   {
         // Process
   }
}

The above code snippets show instantiating, retrieving list of Log Groups, Streams within a particular log group, events within a specified Log Group and Stream.

There are additional parameters that can be specified in the requests. The most important being StartTime, StartFromHead for GetLogEventsRequest, NextToken, LogStreamNamePrefix, OrderBy for DescribeLogStreamsRequest.

Once all the important logs are ingested, an application can be built for monitoring threats or for viewing logs etc… As mentioned above, there will be more posts regarding Cloudwatch, logs, monitoring etc…

Categories
Welcome

Welcome!

This blog is for technical related information. As a one-person company, recently, I have been doing a lot of development and some system administration. This blog is to share code snippets – primarily C#, ASP.Net MVC, Javascript, HTML and some server administration, AWS / Azure cloud administration related knowledge. Once in a while, I might post some R and Python code snippets on a necessary basis, I did dabble with R and Python a few years ago to learn Data Analysis, Machine Learning and A.I. But I did not take up Data Analysis, Machine Learning and A.I as a career, but have some knowledge. This blog serves as a reference for myself and for other .Net developers. In other words, like millions of other people, this blog helps for sharing knowledge. My 2 cents to the blogosphere.