Tuesday, September 24, 2024

2024-09-23 - File Monitoring Powershell (.ps1) script

     In short, I have an Ubuntu file server, connected to a windows desktop, through SMB protocol, so the directories can be viewed as you do on a windows computer. For some reason files seem to disappear from the file server in a particular folder, which I frequent so I know I unwittingly delete things. I wrote a PowerShell script to check this folder for me and check what all the files are in the folder and remember them in a list formed by the script, to then be compared to and updated each time the script runs and specifically tell me when files are missing. The script will create a list of the files and then compare that list to the current contents of the folder each time it runs, alerting me if any files are missing.

    I was going to make it a .bat script, but PowerShell has built in functions that would make it easier. And I had to modify my plan so that I would have two scripts, one to create the list of file contents recursively, and a second script that will check that list against the folder contents later. If I want to update the list I have to re-run the first script. That is just it's current iteration though. Perhaps there is a way to get it to run off one script but I don't know, this was just my instructors suggestion on the matter. 

    The first script I wrote is here:

# Script: CreateFileList.ps1


# Open a dialog box for folder selection

Add-Type -AssemblyName System.Windows.Forms

$folder = [System.Windows.Forms.FolderBrowserDialog]::new()

$null = $folder.ShowDialog()


# Get the selected folder path

$selectedPath = $folder.SelectedPath


# Check if a folder was selected

if (-not [string]::IsNullOrEmpty($selectedPath)) {

    # Define the output CSV file

    $csvFile = "file_list.csv"

    

    # Get the list of all files and folders recursively

    $fileList = Get-ChildItem -Path $selectedPath -Recurse | Select-Object FullName


    # Export the list to a CSV file

    $fileList | Export-Csv -Path $csvFile -NoTypeInformation


    Write-Host "File list created and saved to $csvFile"

} else {

    Write-Host "No folder selected."

}

The second script is here:

# Script: CheckForMissingFiles.ps1


# Open a dialog box for folder selection

Add-Type -AssemblyName System.Windows.Forms

$folder = [System.Windows.Forms.FolderBrowserDialog]::new()

$null = $folder.ShowDialog()


# Get the selected folder path

$selectedPath = $folder.SelectedPath


# Check if a folder was selected

if (-not [string]::IsNullOrEmpty($selectedPath)) {

    # Define the path to the CSV file

    $csvFile = "file_list.csv"

    

    # Check if the CSV file exists

    if (-not (Test-Path $csvFile)) {

        Write-Host "CSV file not found. Please run the script to create the file list first."

        exit

    }


    # Import the original file list from the CSV

    $originalList = Import-Csv -Path $csvFile | Select-Object -ExpandProperty FullName


    # Get the current list of files and folders recursively

    $currentList = Get-ChildItem -Path $selectedPath -Recurse | Select-Object -ExpandProperty FullName


    # Compare the original list with the current list

    $missingFiles = $originalList | Where-Object { $_ -notin $currentList }


    if ($missingFiles) {

        Write-Host "The following files/folders are missing:"

        $missingFiles

    } else {

        Write-Host "No files/folders are missing."

    }

} else {

    Write-Host "No folder selected."

}

I tested the script. When you click it, you have to right-click on it and select to run in PowerShell, because left-clicking it just opens it for editing for reasons I don't yet understand. Then it will open a window that will have you specify what folder you want to take an account of and it will take a moment depending on the size of the folder and how many files it contains, but it will then export a .csv file that will show all the contents of the folder with full names and their file formats. 

    The second script when run didn't seem to produce any results. This is when my instructor was looking at it and checked the end of the second script and realized that it spits out the results but it isn't pausing at the end so I have any time to even register that results were given before they disappear. So we added a pause at the end. 

    So literally just like this for the end of the second script:

} else {

    Write-Host "No folder selected."

}


pause

That resulted in it leaving an open prompt window that now shows that there are many missing files, so technically the two scripts in conjunction with each other worked perfectly. And they work with a networked drive located on a Linux system, from a Windows PowerShell script. So that was a happy result. 

    The problem now is that when the second script compared the results against the .csv of the first script of the file contents where files seem to magically disappear all the time and I need to know which ones, the second script returned results for many missing files that I then went to check to see if they were gone and they were still there. What gives? However I did notice something. 

    A movie file I was supposedly missing but confirmed was not missing on my server was named: "Clint Eastwood Dollar Trilogy I A Fistful Of Dollars 1964 Clint Eastwood Marianne Koch José Calvo.mp4" while the name given to it by the first script which exports the .csv file calls it: "Clint Eastwood Dollar Trilogy I A Fistful Of Dollars 1964 Clint Eastwood Marianne Koch Jos? Calvo.mp4"

    Instead of the accent over the E in 'Jose Calvo', the script or the character base used to create the script didn't recognize it and put "Jos?" instead of Jose with the accent over the E, and so this was counted as one of the missing files from the original list. The second script saw the file with the accent over the letter E on this file and similar discrepancies in all other cases, it saw those correctly named files on the server but is not programmed to tell me that "hey, this file with the question mark wasn't found but one without it was found that was not included in the .csv file", and it also isn't programmed to know the difference. So it didn't see script 1's error with the question mark replacing the E with the accent and then script 2 returned the output saying that the file with the question mark couldn't be found. And it did the same with all other files that it found the same discrepancy with. 
    I emailed my instructor about it and then researched online if this was a problem with the first script not using UTF-8 or Unicode because I know from my A+ Fundamentals certification that ASCII only has like 128 characters or something like that but Unicode intentionally has millions and is still growing in order to accomodate every situation and I found that making this one change to have script 1 utilize Unicode was a very simple change. I added the change, so instead of the line in script 1 saying part way through the script:

    # Export the list to a CSV file

    $fileList | Export-Csv -Path $csvFile -NoTypeInformation

    Write-Host "File list created and saved to $csvFile" 

    It now instead says:

    # Export the list to a CSV file

    $fileList | Export-Csv -Path $csvFile -NoTypeInformation -Encoding UTF8

    Write-Host "File list created and saved to $csvFile"

    I made the change in class and tested it in front of Doug, my instructor, and it returned nothing, the result was blank. I threw up my hands in frustration like, "What is the problem now!?" 
    Before I could even fully express this thought, Doug said, "Oh look, it worked" or something like that and I thought he was being sarcastic and after expressing my frustration, he said, "No, it did work, look, no files were listed in the return, isn't that what we want--for nothing to be missing?"
    "Oh...it worked! I don't believe it!"
    And you can't know all the baggage behind this but I deal with a lot of frustration over various difficulties with things like this, so when Doug said, "And you figured out the problem and fixed it without any of my help", I realized what he was saying and was sort of elated. 
    This technically isn't the end of the story yet though because there are several changes I want to make to these scripts, potentially combining them if possible, configuring them so the second script doesn't require that the .csv from the first script be relocated to the folder in question in order to run the comparison properly, I would like to just specify that it's here and the folder in question is over there. And there are other changes I would like to make too. 

    Thursday 2024-09-26

    I went back into the second script today and made some changes so that the system opens a window and asks which folder I'd like an accounting of as well as as opening another window to specify where the .csv file is located. I changed it to the following:

# Script: CheckForMissingFiles.ps1

# Open a dialog box for folder selection
Add-Type -AssemblyName System.Windows.Forms
$folder = [System.Windows.Forms.FolderBrowserDialog]::new()
$null = $folder.ShowDialog()

# Get the selected folder path
$selectedPath = $folder.SelectedPath

# Check if a folder was selected
if (-not [string]::IsNullOrEmpty($selectedPath)) {
    # Open a dialog box to select the CSV file
    Add-Type -AssemblyName System.Windows.Forms
    $csvFileDialog = [System.Windows.Forms.OpenFileDialog]::new()
    $csvFileDialog.Filter = "CSV Files (*.csv)|*.csv"
    $null = $csvFileDialog.ShowDialog()

    # Get the selected CSV file path
    $csvFile = $csvFileDialog.FileName

    # Check if a CSV file was selected
    if (-not [string]::IsNullOrEmpty($csvFile)) {
    }

    # Import the original file list from the CSV with UTF-8 encoding
    $originalList = Import-Csv -Path $csvFile -Encoding UTF8 | Select-Object -ExpandProperty FullName

    # Get the current list of files and folders recursively
    $currentList = Get-ChildItem -Path $selectedPath -Recurse | Select-Object -ExpandProperty FullName

    # Compare the original list with the current list
    $missingFiles = $originalList | Where-Object { $_ -notin $currentList }

        if ($missingFiles) {
            Write-Host "The following files/folders are missing:"
            $missingFiles
        } else {
            Write-Host "No files/folders are missing."
        }
    } else {
        Write-Host "No CSV file selected."
    }
pause

I had to have Doug's help to make a few edits because it broke before it even reached the pause at the end of the script. I had a moment where I thought I deleted a curly que bracket and so I put what I thought I deleted back in and that was one of the things that broke the script. 
    I am now thinking of changing what the line at the end says which tells me that the script finished and no files were missing, and I had the idea to do something like ASCII art except this is Unicode I'm using PowerShell. So I am going to try to write the echo command on every line and create some sort of unmistakable image that when I see it, it will signal to be that I am done without any confusion. 


        Saturday 2024-10-19

        Remember how this all came around because I would occasionally discover files missing from folders I realized were usually commonly accessed folders? I had been thinking about what can be done to minimize the kind of traffic on folders or files that I definitely don't want deleted but due to high traffic, unwittingly get deleted by accident. And the solution for my mom was simple--that she not have write access to her own movies folder so she can't delete anything, and then to make it easier for me to put movies in her folder for her, I linked her movies directory in Linux to my movies directory so I can easily add things or whatever, but then I decided to just put her directory in mine and then relegate her access to just my movies and her movies, and by sharing our movie folders with each other I can eliminate unwanted, space hogging copies. And then the idea hit me: what if I put all of my files in a folder that I likewise will also not be accessing hardly ever except to put files into it, which is far rarer, nearly eliminating the traffic, and then in Windows, create shortcuts in the folder directly before it in the file path? 
    So I did this. It worked. Now I can access everything and if a shortcut gets deleted, it will still be annoying but at least I can keep track of it now with this file monitoring script file and then just go into the folder with all the actual files and grab another copy to make a shortcut with. 
    I had a moment where my heart fell through the floorboards when I opened a common file that also had common association with another file, and I dropped the secondary file into the program that runs all these files to see if the shortcut of the secondary file would still work and it did not. I then considered that since these secondary files are so incredibly small it's not a big deal to just bite the bullet and have redundant copies of those in the above folder of the file path, but then I realized the reason it wasn't working wasn't because the program couldn't open the file for the shortcut in the above folder, but rather because I had a bunch of computer errors that were annoying me, and had just rebooted, and since I seldom reboot unless I have to due to all the clockwork I have going on with my really stupid smart TV and AV receiver and my desktop and the slightest little thing will cause my smart tv to forget that I want my audio to come from my computer and simply play out of the speakers connected to the AV receiver and not my TV. All of that to say that the reason the secondary file shortcuts weren't working was all because of rebooting my PC, I had forgotten that I needed to log back in to SMB to access my file server. And there, it worked fine. 





















This has been Truncat3d 00000000111100010100110______________end of line


No comments:

Post a Comment

2025-07-10 - BYU Wi‑Fi captive portal troubleshooting

  BYU Wi‑Fi Captive Portal Troubleshooting What happened I had used BYU Wi‑Fi just fine before, but at some point, the captive portal star...