Tuesday 21 February 2023

Powershell to Restore Azure SQL database PITR backup

Today there seems to be a bug on Portal resource Manager API not allowing any SQL databases pitr restores from portal and the only way to do it at this point is using scripts, The below script is an example to restore the database on the same server with Feb 20th 3 PM backup

Copyright © 2023 Vinoth N Manoharan.The information provided in this post is provided "as is" with no implied warranties or guarantees.

Power BI DAX for Start of Year To End of Prior Month Sales

We had a requirement for Power BI Measure Start of the year To End of Prior Month Sales sum for one of the report which was failed for January as the original Measure was just query Previous Month and going to a negative Data Diff value obviously for January Eg:DATEDIFF(1jan2023,31DEC2022). To avoid this we had to introduce a check on the measure if the month is january then we need SUM of Sales of January rather than previous Month. I used the below logic to solve this issue
Copyright © 2023 Vinoth N Manoharan.The information provided in this post is provided "as is" with no implied warranties or guarantees.

Bug In Azure API creating Data Factory Default AutoResolveIntegrationRuntime on Managed Virtual Network

Recently our terraform Infrastructure as code effort to create a datafactory resource with default AutoResolveIntegrationRuntime in Managed Virtual Network failed to create one.

We had managed_virtual_network_enabled flag on terraform and tried to use native azure cli too as a backup but both failed to create the default integration runtime on Managed Virtual Network and created a default public integration runtime. Only work around it was to create a Data Factory using ARM Template. I exported the ARM template and created the below Powershell to create the Data factory. I stored the Template and parameter file on a Fileshare and used it in the powershell to reuse and create a parameter file each time for different DF creation.
Copyright © 2023 Vinoth N Manoharan.The information provided in this post is provided "as is" with no implied warranties or guarantees.

Powershell Azure Devops REST API to create/update a variable group based on json variable input

 I had a requirement to automate creation and updating of Azure devops variable group based on a configuration variable json file proved to me by the application team for CICD process, the below script will look for the variable group if it exists with the project mentioned if not it will create a new variable group, If the variable group exists it will update the variable group with json provided. The idea is to maintain the json file on git for version and see the history of the changes on the variable group variables provided to the pipeline.

The below is the format of the Variable json passed on to the script

Copyright © 2023 Vinoth N Manoharan.The information provided in this post is provided "as is" with no implied warranties or guarantees.

Thursday 3 November 2022

Work with Azure SQL with Python import result to pandas dataframe

I am going start a series of blogs working with python on Azure SQL, what we can do and limitations, In this first blog we look look into connecting to a database and importing a sql result to pandas dataframe. In the below example we can see we have imported the data and printed the last 10 rows and I have attached the screen shot of the result. In the future blogs we can se how we can work in pandas with the data.



Wednesday 9 February 2022

Azure Windows VM Disk Usage report for All VM in a Azure Tenant

I was setting up monitoring for my Azure environment and I had an requirement to email the VM Disk usage of all windows VM's in a tenant to Support team on daily Basis and also sent notification in Slack. I used PowerShell to consolidate the data and Logic apps for the integration for Slack notification.


To breakdown the script for better understanding, I have used a credential from credential manager which had RBAC permissions to all subscriptions in the tenant and I used the function in  https://gallery.technet.microsoft.com/scriptcenter/Accessing-Windows-7210ae91 for my Azure and SFTP authentication using the credentials already in credential manager. I am using Invoke-AzVMRunCommand which basically runs a PowerShell script file on the remote server. The below PowerShell script will run C:\temp\GetDisk.ps1 on all the remote servers I am passing. 

GetDisk.ps1 runs the below PowerShell commandlet 

"Get-Volume|where {$_.FileSystemLabel -ne 'System Reserved' -and $_.FileSystem -eq 'NTFS'}|SELECT DriveLetter,Size,SizeRemaining|ConvertTo-Json"

Invoke-AzVMRunCommand Will return only as single string irrespective of output format so I have converted into json so I can easily convert to PSObject for processing in my main script.


The script will consolidate result in HTML format and Json format. Json format will be sent as Payload to Webhooks to Logic Apps were you can integrate with Slack or Teams alerts. HTML format will be styled with yellow rows for 10 to 15% free space and Red Rows for usage less than 10% free space as below.

VM Disk Usage

VMName

SubscriptionName

RGName

DriveLetter

Size(GB)

SizeRemaining(GB)

UsedSpace(GB)

FreeSpacePercentage

Server1

Sub-1

RG-1

C

512

223

289

43.49978

Server1

Sub-1

RG-1

D

32

30

2

92.29322

Server1

Sub-1

RG-1

F

2048

381

1667

18.60302

Server2

Sub-2

RG-2

C

127

19

108

14.84652

Server2

Sub-2

RG-2

F

512

27

485

5.293138

 

Copyright © 2022 Vinoth N Manoharan.The information provided in this post is provided "as is" with no implied warranties or guarantees.

Azure File Share Usage Report for all Subscription in your Tenant (Email and Web hooks notification)

 I was setting up monitoring for my Azure environment and I had an requirement to email the Fileshare usage of all fileshares in a tenant to Support team on daily Basis and also sent notification in Slack. I used PowerShell to consolidate the data and Logic apps for the integration for Slack notification.

To breakdown the script for better understanding, I have used a credential from credential manager which had RBAC permissions to all subscriptions in the tenant and I used the function in  https://gallery.technet.microsoft.com/scriptcenter/Accessing-Windows-7210ae91 for my Azure and SFTP authentication using the credentials already in credential manager. I am using Azure Management API's to get the information of the File storage as I found PowerShell Modules had restriction to Storage account opened only for particular subnet outside of the server I am running the script. I have used the method in  https://www.powershellgallery.com/packages/LSECosmos/0.2.1-alpha/Content/Get-AzAccessToken.ps1 for API Authentication.


The script will consolidate result in HTML format and Json format. Json format will be sent as Payload to Webhooks to Logic Apps were you can integrate with Slack or Teams alerts. HTML format will be styled with yellow rows for 80 to 90% and Red Rows for usage above 90% as below.

File Share Usage

StorageAccountName

SubscriptionName

ResourceGroupName

ShareName

TotalCapacity(GB)

UsedSpace(GB)

PercentageUsed

StgAcctName1

Sub-1

RG-1

FileShare-1

1

0

0

StgAcctName2

Sub-1

RG-2

FileShare-2

10

0

2

StgAcctName3

Sub-2

RG-3

FileShare-3

100

0

0

StgAcctName4

Sub-3

RG-4

FileShare-4

1024

926

90

StgAcctName5

Sub-4

RG-5

FileShare-5

250

91

37

StgAcctName6

Sub-4

RG-5

FileShare-6

1024

845

83

 

Copyright © 2022 Vinoth N Manoharan.The information provided in this post is provided "as is" with no implied warranties or guarantees.