How could we extract desired string's from a text file by regex pattern search

In this article I would like to share a tips to copy a desired block of text's from a large text file. To explain this I am using Notepad++ and you need to basic knowledge about how the regex work.

Say I have a text file like below..

DTP*472*RD8*20150908-20150908~ TRN*2*435577~ STC*A2:20:PR*20160906*WQ*4027.2~ REF*1K*1625000104243~ REF*BLT*131~ DTP*472*RD8*20150609-20150609~ TRN*2*447535~ STC*A2:20:PR*20160906*WQ*436.8~ REF*1K*1625000104244~ REF*BLT*131~ DTP*472*RD8*20150713-20150713~ TRN*2*500331~ STC*A2:20:PR*20160906*WQ*3941.6~ REF*1K*1625000104245~ REF*BLT*131~ DTP*472*RD8*20150812-20150812~ TRN*2*484995~ STC*A2:20:PR*20160906*WQ*436.8~ REF*1K*1625000104246~ REF*BLT*131~ DTP*472*RD8*20150730-20150730~ HL*18*3*PT~ NM1*QC*1*ROSS*RICHARD****MI*334589625A~TRN*2*354771~STC*A7:673*20160906*U*8613.98~STC*A7:509*20160906*U*8613.98~TRN*2*516925~STC*A2:20:PR*20160906*WQ*4204.23~ REF*1K*1625000104247~ REF*BLT*131~ DTP*472*RD8*20150427-20150427~ TRN*2*361769~ STC*A7:460*20160906*U*434.2~ REF*BLT*121~ DTP*472*RD8*20150423-20150423~ TRN*2*268379~ STC*A7:255*20160906*U*25952.66~ REF*BLT*111~ DTP*472*RD8*20150209-20150212~ HL*19*3*PT~ NM1*QC*1*BLACKBURN*LALAH****MI*326388373A~ TRN*2*397463~ STC*A7:673*20160906*U*2400.3~

And I need to extract only those values that are indicated by red color. How we could achieved this?
Yes its time to explain step by step how it's work.

Note: Make sure when you copy any text from here, trim the text first to search in notepad++

1.  Copy the above test to notepad++
2. Now press Ctrl + F5 to open search windows.
3. Now try to find a pattern to search desired string's. (In this case the pattern is TRN*2*xxxxxx*~. Regex search expression for this pattern is (TRN\*2\*)([0-9]+)([~])
4. Now we need to break down all matched pattern in new line. To do this press Ctrl+H. Then write the regex expression (TRN\*2\*)([0-9]+)([~]) to Find what field and fill \n$1$2$3 to Replace with field. $1$2$3 indicate the consecutive group in above regex expression. (you need to put your own regex expression and group). Make sure you select Regular expression radio button in Search Mode block in replace window. And again make sure your cursor point at the first character of your text.
   



5. Now press Replace All button and the text look like below image. 
6. 
7. Now change the regular expression from (TRN\*2\*)([0-9]+)([~]) to (TRN\*2\*)([0-9]+)([~]).* and click Replace All button again.
   

   
8. Now the text look like below image.
   

   
9. Now click on Mark tab from replace window and checked the Bookmark line  checkbox.
   

   
10. Now click Mark All button and your text look like below image.
    

    
11. Now from Search menu go to Bookmark and click on Remove Unmarked Lines. It will remove all unmarked lines and the file looks like.
    

    
12. Now click on Replace tab again and change the value of Replace with field by $2 which mean second group of regular expression.
    

    
13. Now click on Replace All  button.
    

    

Here we achieved our desired value from the above text.

Thanks for the patience to go through the whole article. Any feedback or comments will more appreciated

How to transform different config file without any third party extension.

Introduction

This article shows how to automate the process of transforming the configuration file like App.config, Web.config file when we deploy it to the different destination environments like Dev, Stg, Prod. Most applications have settings in the App.config or Web.config file that must be different when the application is deployed. Automating the process of making these changes saves us from having to do them manually every time we deploy, which would be tedious and error prone.

Background

In most real-world scenarios, the configuration (app.config, web.config) file we use for development is different than the one we use for production deployment. Typically we want to change environment settings from the development environment to the production environment. From .Net 4.0 XDT Transformation is come into play to do this type of transformation.

How transformation work?

Create a console app 'ConfigurationTransform' from visual studio.

• From solution configuration menu click 'Configuration Manager'
  
• From Configuration manager click 'Active solution configuration' drop down and click new.
  
• Set the name of the configuration like 'Dev' and select Copy settings from as 'Release' and then click ok.
  
• Open project directory by right-clicking on the project and click 'Open Folder in File Explorer'
  
• Now copy the App.config file and paste in the same directory and rename it to App.Dev.config.
• Unload the project by right-clicking on the project and click 'Unload project'. If save change prompt appeared then click yes.
  
• Right-click on the project and click on 'Edit ConfigurationTransform.csproj'
  
• Find the Item group 'None Include="App.config"' and add another code block just below that as
  Hide   Copy Code

      <None Include="App.Dev.config">
        <DependentUpon>App.config</DependentUpon>
      </None>

• Now add this block of code just before '</Project> tag. We need to run this 'Microsoft.Web.Publishing.Tasks.dll' MSbuild task to do transformation.
  Hide   Copy Code

    <UsingTask TaskName="TransformXml" AssemblyFile="$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion)\Web\Microsoft.Web.Publishing.Tasks.dll" />
    <Target Name="AfterCompile" Condition="Exists('App.$(Configuration).config')">
      <!--Generate transformed app config in the intermediate directory-->
      <TransformXml Source="App.config" Destination="$(IntermediateOutputPath)$(TargetFileName).config" Transform="App.$(Configuration).config" />
      <!--Force build process to use the transformed configuration file from now on.-->
     <ItemGroup>
        <AppConfigWithTargetPath Remove="App.config" />
        <AppConfigWithTargetPath Include="$(IntermediateOutputPath)$(TargetFileName).config">
          <TargetPath>$(TargetFileName).config</TargetPath>
        </AppConfigWithTargetPath>
      </ItemGroup>
    </Target>

• Now reload the project by right-clicking on the project and click on 'Reload Project'
  

Now add appsettings block in configuration section in App.config as 

Hide   Copy Code

<appSettings>
    <add key="adminEmail" value="admin@local.com" />
    <add key="serviceUri" value="Local ServiceUri" />
  </appSettings>

Add xml namespace attribute 'xmlns:xdt' in App.Dev.config file as 

Hide   Copy Code

<?xml version="1.0"?>
<!-- For more information on using app.config transformation visit http://go.microsoft.com/fwlink/?LinkId=125889 -->
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <appSettings>
    <add key="adminEmail" value="admin@dev.com" xdt:Transform="Replace" xdt:Locator="Match(key)" />
    <add key="serviceUri" value="Dev ServiceUri" xdt:Transform="Replace" xdt:Locator="Match(key)" />
  </appSettings>
</configuration>

Points of Interest

Now build the solution and go to ../bin/Dev folder and click on 'ConfigurationTransform.exe'. It will print transformed configuration value as

You can add more configuration for different environment like QA, Stg, Prod.

How could we mock a non-virtual method from an external dependency.

Please choose 'View Source' in your browser to view the HTML, or File | Save to save this file to your hard drive for editing.

---

Introduction

Through this article, we are knowing about how we able to mock a non-virtual method with no interface implementation of an external object. To understand this article you need hands-on unit-testing experience in c# with Machine.Specifications.

Mocking is not easy always

Most often when we are writing unit test, we are stuck with some not-virtual method which could not be easily mocked. Through this article, I am trying to explain how could we achieve this by below code snippet.

Consider a situation when we want to upload a file to AWS (Amazon Web Services) S3 location,  we are using AWS SDK which exposes a TransferUtility class, a simple API for uploading content to S3.
This blocks of code should be used to upload files to S3 location and this.clientFactory.Get(request.AwsIdentifier) method return and AmazonS3 object. Here if you closely look into the uploader object, this is instantiated by TransferUtility class with a constructor which receive a signature of AmazonS3.

    public class Uploader : IUploader
    {
        public Uploader(IClientFactory clientFactory)
        {
            this.clientFactory = clientFactory;
        }
        
        private readonly IClientFactory clientFactory;
        
        public async Task<string> Upload(UploadRequest request)
        {
            using (var client = await this.clientFactory.Get(request.AwsIdentifier))
            {
                using (var uploader = new TransferUtility(client))
                {
                    if(uploader == null)
                    {
                        throw new ArgumentException("Uploader is required", "fileLocation");
                    }
                    var awsRequest = new TransferUtilityUploadRequest
                    {
                        BucketName = request.BucketName,
                        Key = path,
                        FilePath = request.FileLocation,
                     };

                    await uploader.UploadAsync(awsRequest);
                }

                return request.BucketName + request.FileLocation;
            }
        }
    }

Points of Interest

If we need to write a unit test for method uploader.UploadAsync(awsRequest) to check this method received the valid value which we provided by calling Upload(UploadRequest request).

    [Subject(typeof(Uploader))]
    public abstract class When_upload_a_file
    {
        Establish context = () =>
        {
            s3ClientFactory = Substitute.For<IClientFactory>();
            amazonS3 = Substitute.For<IAmazonS3>();

            uploader = new Uploader(s3ClientFactory);
            
            
            uploadRequest = new UploadRequest
            {
                BucketName = "example-bucket-name",
                AwsIdentifier = "12345",
                FileLocation = "abc.txt"
            };
        };
        
        Because of = async () =>
        {
            response = await uploader.Upload(uploadRequest);
        };

        
        It transferUtility_should_reveive_bucket_name_in_uploader_request =
            () => uploader.Received().Upload(Arg.Is<TransferUtilityUploadRequest>(r => r.BucketName == uploadRequest.BucketName), Arg.Any<CancellationToken>());

        
        
        protected static Uploader uploader;
        protected static IAmazonS3 amazonS3;
        protected static IClientFactory s3ClientFactory;
        
        static UploadRequest uploadRequest;
        static string response;
    }

Complexity arise here!, We could not achieve uploader proxy because uploader objects instantiated by the concrete implementation of TransferUtility class.  TransferUtility doesn't implement an interface and the method I need to mock is not virtual.  that's why we could not mock this object for calling uploader.UploadAsync(awsRequest) method.

How could we achieve this?

If we face like this situation where we are not able to change the external API class, we just need to introduce a wrapper class to avoid concrete class. Below code snippet is an interface and its implementation which have a UploadAsync method with the same signature that contains in UploadAsync method of TransferUtility class. And implemented wrapper class have a constructor which get original TransferUtility object.

 public interface IAwsTransferUtilityWrapper : IDisposable
    {
        Task UploadAsync(TransferUtilityUploadRequest request, CancellationToken cancellationToken = default(CancellationToken));
    }
    public class AwsTransferUtilityWrapper : IAwsTransferUtilityWrapper
    {
        private readonly TransferUtility originalTransferUtility;

        public AwsTransferUtilityWrapper(TransferUtility originalTransferUtility)
        {
            this.originalTransferUtility= originalTransferUtility;
        }

        public async Task UploadAsync(TransferUtilityUploadRequest request, CancellationToken cancellationToken = new CancellationToken())
        {
            await this.originalTransferUtility.UploadAsync(request, cancellationToken);
        }

        public void Dispose()
        {
            this.originalTransferUtility.Dispose();
        }
    }
}

We need to change the instantiation of uploader object in Uploader class as below code snippet

    public class Uploader : IUploader
    {
        public Uploader(IClientFactory clientFactory)
        {
            this.clientFactory = clientFactory;
        }
        
        private readonly IClientFactory clientFactory;
        public static  Func<IAmazonS3, IAwsTransferUtilityWrapper> AwsTransferUtilityFactory { get; set; } = client => new AwsTransferUtilityWrapper(new TransferUtility(client));
        
        public async Task<string> Upload(UploadRequest request)
        {
            using (var client = await this.clientFactory.Get(request.AwsIdentifier))
            {
                using (var uploader = AwsTransferUtilityFactory(client))
                {
                    if(uploader == null)
                    {
                        throw new ArgumentException("Uploader is required", "fileLocation");
                    }
                    var awsRequest = new TransferUtilityUploadRequest
                    {
                        BucketName = request.BucketName,
                        Key = path,
                        FilePath = request.FileLocation
                    };

                    await uploader.UploadAsync(awsRequest);
                }

                return request.BucketName + request.FileLocation;
            }
        }
    }

Now we can test our uploader.UploadAsync(awsRequest) method as like below unit test.

    [Subject(typeof(Uploader))]
    public abstract class When_upload_a_file
    {
        Establish context = () =>
        {
            originalTransferUtilityFactory = Uploader.AwsTransferUtilityFactory;
            transferUtility = Substitute.For<ITransferUtilityWrapper>();
            
            s3ClientFactory = Substitute.For<IClientFactory>();
            amazonS3 = Substitute.For<IAmazonS3>();

            Uploader.AwsTransferUtilityFactory = _ => transferUtility;
            uploader = new Uploader(s3ClientFactory);
            
            
            uploadRequest = new UploadRequest
            {
                BucketName = "example-bucket-name",
                AwsIdentifier = "12345",
                FileLocation = "abc.txt"
            };
        };
        
        Because of = async () =>
        {
            response = await uploader.Upload(uploadRequest);
        };

        
        It transferUtility_should_reveive_bucket_name_in_uploader_request =
            () => uploader.Received().UploadAsync(Arg.Is<TransferUtilityUploadRequest>(r => r.BucketName == uploadRequest.BucketName), Arg.Any<CancellationToken>());

        
        
        protected static Uploader uploader;
        protected static IAmazonS3 amazonS3;
        protected static IClientFactory s3ClientFactory;
        protected static IAwsTransferUtilityWrapper transferUtility;
        static Func<IAmazonS3, IAwsTransferUtilityWrapper> originalTransferUtilityFactory;
        
        static UploadRequest uploadRequest;
        static string response;
    }

Now we are able to  mock a method of external API class by imploementing a wrapper class 'AwsTransferUtilityWrapper'.

Manage Entity Framework Code First Migration in Version Control Environment

How to merge and manage Entity Framework code first migration, when a different team work on the same application with a source control environment.

Introduction

I know there are several articles on Entity Framework Code First approach. But This article intended for those, who face problem to execute Update-Database command in entity framework  after merging the different branch with the change in the same domain object.

Background

There are many contents available in the web to demonstrate how Entity Framework Code First work and I assumes you know that and this is the prerequisite to grab the abstract of this article. If you don't then you can dig into the links:

• http://www.asp.net/mvc/overview/getting-started/getting-started-with-ef-using-mvc/creating-an-entity-framework-data-model-for-an-asp-net-mvc-application
• http://www.asp.net/mvc/overview/older-versions/hands-on-labs/aspnet-mvc-4-entity-framework-scaffolding-and-migrations
• https://msdn.microsoft.com/en-us/data/jj591621

So I'd not like to re-invent above. This tutorial may assist when you face problem to execute Update-Database command in Package Manage Console.
When we manage a source repository and work in a different team, Entity Framework may cause a problem when two developer change in same domain object in their local code base and Add-Migration to local database and then push code to remote branch.
I'll demonstrate how to generate above situation step by step and how we can resolved it.

Scenario to generate migration problem

Considering that, In a development team, team 'X' works on his local code base with local database and team 'Y' also work on his local code base and local database. Team 'X' add a column to a domain object "Student" and add-migration '***_Address column added to student table' and update-database in locally. On the other hand Team 'Y' also add a column to same "Student" domain object and add-migration '***_EnrollmentDate column added to student table' and update-database locally.

Now team 'X' commit their local change and push to remote repository. And team 'Y' take a pull from the remote branch and get all changes that are done by team 'X' and resolved the conflict.

Points of Interest

Now when team 'Y' build the solution and try to execute command 'Update-Database' in package manager console. Stop! for a while and see what happen. All database change is done perfectly but package manager console shown bellow message:

This is because Entity Framework notice that domain model snap shot has been changed by pulling and merging code base from remote branch.

How could we handle this situation

It's time to add a blank migration to update snap shot of current domain model for team 'Y'. To add a blank migration team 'Y' need to add 'IgnoreChanges' attibute in package manager console. One more things, team 'Y' need to consider a name for this migration which was never used in this solution.

Now execute Update-Database command in package manager console and everything is done to handle this situation. Now time to commit and push this code to remote branch, So that other team could use this code base without any hastle.