Compiling NodeMCU ESP32 Firmware

Recently I’ve been doing some experimenting with the NodeMCU ESP8266 and ESP32-S chips to use in home automation, sending data back to my Samsung SmartThings system. The ESP32 is considered a development board and as such not everything works as expected. One of the issues I was having was the ADC isn’t very linear resulting in some of my analog inputs being pretty far off (over 4%). With that said the firmware is being updated pretty regularly so I wanted to compile a new version. I attempted to follow the instructions available (https://nodemcu.readthedocs.io/en/dev-esp32/en/build/) but was having trouble. Finally I created a Ubuntu 16.04.3 LTS virtual machine using VirtualBox but then ran into a bunch of other issues since it didn’t have any of the dependencies installed. Well after some experimenting I got it working and here are the steps:

  1. Install Ubuntu 16 (should work with 17 also)
  2. Update it (search for update and launch the software updater)
  3. Drop to a terminal (CTRL+ALT+T), typing each item, and press enter:
    • sudo apt-get update
    • sudo apt-get upgrade
    • sudo apt-get install libncurses5-dev libncursesw5-dev flex bison gperf python-serial
    • sudo apt-get install git
  4. In the same terminal window grab the firmware code: “git clone --branch dev-esp32 --recurse-submodules https://github.com/nodemcu/nodemcu-firmware.git nodemcu-firmware-esp32
  5. Move to the source directory with “cd nodemcu-firmware-esp32”
  6. Start up the menu using “make menuconfig”

You should now be able to select your config options and save a sdkconfig file that you can use to make the firmware. Now I couldn’t save the sdkconfig in the same directory and ended up saving it to my home directory then making a copy of the modemcu-firmware-esp32 directory, pasting the config in there, and running make from there to compile. I’m assuming this is because I used git to pull down the repo and its read only (compiling from the copy with my sdkconfig worked so I didn’t try to figure it out).

If you want to get fancy you can also share out your USB through VirtualBox and flash the chip from the virtual machine using make flash but I didn’t like that idea so I transferred out the NodeMCU.bin from the build directory to my host machine (Windows 10) and used NodeMCU-PyFlasher-2.0 to flash to firmware. Afterward I found that my ADC’s, while not 100% accurate, were a lot closer. Hopefully they keep making progress on this chip as it seems like a very capable replacement to the ESP8266.

Installing Pydio 8 on IIS 8.5 and IIS 10

Pydio 8.0.0 was released earlier today and has a major UI change along with some minor back-end changes. I’ve gone through and did another re-write of the instructions for this new version. Among other things it now supports PHP 7.1.*. I’ve also streamlined the instructions slightly, upgrading all other pre-reqs to the latest versions, and use the modified version of PHP Manager that Ronald Carter has upgraded to no longer require Dot Net 2.0 so less stuff to install on the server. So it should now work without issues on both IIS 8.5 and IIS 10. Here are the new instructions:

Installing Pydio 8 on Windows Server 2012 R2

Installing Pydio 7 on IIS 8.5 – Pure 64 bit

With the recent release of Pydio 7.0 my old instructions are now outdated, especially when it comes to public file shares. There were a lot of changes in the new Pydio so I have re-written my instructions for it. The biggest change is there is no longer any reason to create your own web.config files, the stock one now works correctly on IIS. I’ve also added instructions on scripting things form the command line like indexing workspaces on a schedule which has helped me immensely since I keep some workspaces updated with DFS and have gotten away from using Samba. Here are the new instructions, let me know if you have trouble with them:

Installing Pydio 7 on Windows Server 2012 R2 – Pure 64 bit

Sophos XG Firewall PCI Compliance Woes

Recently I deployed a Sophos XG firewall to replace my very much aging Microsoft Forefront TMG 2010 firewalls. For the most part, after lots of back and forth with Sophos tech support, I got it working correctly for things like Outlook Anywhere, Web and Mobile Access, protecting internal websites, and general web filtering. Everything seemed fine until our monthly PCI compliance scans came along and we failed miserably. TLS 1.0 was enable, HTTP Track/Trace was enabled, and 64 bit cyphers were enabled for each external IP that we were hosting a site on. I contacted Sophos and long story short there is currently no way to fix these through the UI (v16)….all required manually editing the appache httpd file on the box. So here is how to do it.

First telnet into your XG, log in, then go to 5 Device Management then 3 Advance Shell. Type in the following:

# mount -no remount,rw /
# vi /usr/apache/conf/httpd.conf

You should now be able to edit the file. Press “I” once to enter insert mode. Find the SSLCipherSuite line and remove any sections (between the colons) that have 3DES in them. For example the original cypher line:

ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:ECDH+3DES:DH+3DES:RSA+3DES:!aNULL:!MD5:!DSS

Modified cypher line:

ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS

Now find the SSLProtocol list, usually right below the cyphers and remove support for whichever protocol by adding it with a minus. Here is the original line:

SSLProtocol all -SSLv2 -SSLv3

Modified protocols line:

SSLProtocol all -SSLv2 -SSLv3 -TLSv1 -TLSv1.1

Lastly for the tracking add a new line under these:

TraceEnable off

Hit ESC then :w and enter to write the file then :q and enter to quit. Then restart the services and mark things what I’m assuming is read only:

# service apache:restart -ds nosync
# service WAF:restart -ds nosync
# mount -no remount,ro /

Keep in mind any Outlook 2010 clients you have in the field might try to connect using TLS 1.0 and will fail with a cryptic proxy server error. You can follow this site to hopefully fix that: https://blogs.technet.microsoft.com/schrimsher/2016/07/08/enabling-tls-1-1-and-1-2-in-outlook-on-windows-7/

Getting Broadcom Wireless Working on Ubuntu 14 (Dell Inspiron 1501)

Recently we changed out a bunch of rarely used training PC’s, Dell Inspiron 1501’s, for newer models. The Dells were definitely past their prime but I’d hate to throw them out since they all looked good and had almost no wear. From previous memory ad hard drive upgrades I had a stack of DDR2 memory and some decent mechanical drives so I decided to upgrade all of them as much as possible and find a OS that would run on them.

The best I could muster was 1.5Gb of RAM each and some 160Gb 7200 RPM drives. After a quick search it looked like Ubuntu 14 was my best bet (16 has a 2Gb memory minimum). I created a Ubuntu 14 USB drive using Universal USB Installer (http://www.pendrivelinux.com/universal-usb-installer-easy-as-1-2-3/), hooked up the ethernet cord, and booted it. At the install screen I selected the option to download updates and also to isntall third party software. Little did I know that third party software checkbox was going to cause me lost of headaches.

Turns out some firmware for the Broadcom cards is installed by that option which renders the wired and wireless connection in these laptops dead. After lots and lots of searching I found someone that posted how to remove it. Unfortunately there steps didn’t work for me but eventually I found ones that did.

Before I fixed the wireless/wired issue I found that these machines freeze during a shutdown or reboot. Some searching for that issue led me to here http://askubuntu.com/questions/523638/why-does-ubuntu-freeze-during-reboot-14-04-lts and editing the grub config file (CTRL+ALT+T for terminal) with:

sudo gedit /etc/default/grub

And changing these two lines as listed:

GRUB_CMDLINE_LINUX_DEFAULT=”quiet splash reboot=acpi”
GRUB_CMDLINE_LINUX=”acpi”

Once thats fixed run the following to remove the current Broadcom stuff which doesn’t seem to work on this model (or others around the same vintage):

sudo apt-get remove –purge bcmwl-kernel-source

Reboot and the wired connect should be working again. It might not reboot because the above grub changes aren’t yet in effect so let it sit for a minute on the shutdown screen then hard power it off and back on. Now to get the firmware for the wireless card and then reboot again:

sudo apt-get install firmware-b43-installer
sudo apt-get install linux-firmware-nonfree

Now both the wired and wireless should work. And I have decent working Ubuntu machines that we can give to someones kinds to mash on instead of throwing them in the garbage.

Disabling Dropbox from Installing or Running if Installed

Recently I was on a quest to disable the Dropbox program from running on company owned (domain joined) machines. There were lots of hacks to make it work but finally I found a solution, although it was worded relatively cryptically, on Experts Exchange by a McKnife (http://tinyurl.com/gr3f9ar). Long story short you can use Software Restriction Policies (https://technet.microsoft.com/en-us/library/bb457006.aspx) to do this but his solution was more elegant as it blocked Dropbox programs based on the certificate used to sign them as opposed to the file path or things that might change often. This not only blocks the Dropbox program if it’s already installed but also prevents a user from installing it in the first place. Here is my expanded version of his instructions.

First download the Dropbox installer. Right click it and select Properties then go to Digital Signatures. Select the first one (SHA1) and click “Details”. Click “View Certificate” then the Details tab then “Copy to File…”. This lets you export out the certificate. Click Next then “Base-64 encoded X.509 (.CER)” and next again. Save the certificate as something like “Dropbox SHA1 Cert.CER”. Once that one is exported repeat the procedure for the SHA256 certificate.

Once you have both certificates open up Group Policy Management and if you already have a software restrictions policy edit it. If not I suggest you create a new one. Navigate to Computer Configuration -> Policies -> Windows Settings -> Security Settings -> Software Restriction Policies -> Additional Rules. Right click and create a “New Certificate Rule”. Browse for the SHA cert and make sure the Security Level is set to Disallow. Give it a description such as “Dropbox SHA Certificate”. When you click OK, if you didn’t have any certificate rules before, it will prompt you to turn them on and display the “Enforcement Properties” page. At the bottom “Enforce certificate rules” then “OK”. Repeat for the SHA256 certificate.

Once GPO updates Dropbox will no longer start and executing the exe or installer directly will give you a nice error message:

dropboxerror

Side note: Once this policy is in place you will also not be able to uninstall Dropbox since the same certificate is being used on the uninstall. Keep that in mind…you would have to disable enforcing certificates temporarily to get it uninstalled.

Samba Client for Windows – smbclient.exe – v3.6.25

So in reference to getting Pydio working on a Windows Server 2012 R2 machine with IIS 8.5 I found that I wanted a workspace in Pydio to access a existing share on a 2012 R2 file server. I have a lot of files I don’t want duplicated on both servers. Now there are many workspace drivers available but since Pydio was designed originally for a *nix system the main way to access a Windows server is through Samba. Windows Server supports SMB, server message blocks, for file sharing among other things. Now on a *nix system you can just install Samba and then Pydio can use the Samba client to remotely access the Windows server. Problem is I want to use Samba on a Windows server to access another Windows server. So I started a hunt to find a ported version of the Samba client and it resulted in two pages out of the millions out there: https://smithii.com/samba which had ported all of Samba v3.0.23 over and https://www.leepa.io/lpackham/smbclient/ which had ported just the v3.0.7 client. So I started testing these clients and long story short they have issues with Windows servers. Mainly you can get directory listings using them and even pull files but you cannot create files nor push files to the Windows file server. After some research its due to changes, mainly security related, in the Windows servers SMB implementation….changes that were fixed in newer versions of Samba (https://www.samba.org/).

So I determined that the only possible way to get my Pydio install to access my existing Windows file server was to either use the FTP over SSH plugin and enable this on my file server, which I didn’t want to do, or figure out a way to compile the Samba client using the latest source. After lots of trail and error I was able to partially compile Samba 3.6.25 using Cygwin (https://www.cygwin.com/) but where it failed was after it had built the smbclient.exe that I needed. I took the client and copied it to the server and tried running it from a command prompt and it complained about needing some Cygwin DLL files which was expected. I copied over the DLL files it needed to the server and was able to connect to the file server. I then updated Pydio to use my newly compiled smbclient.exe and Pydio now works as expected allowing e to share out a existing share from another server. The best part is since user credentials are passed through the existing NTFS permissions still apply on top of Pydios own permission checks.

Not sure if anyone in the world will ever need it but just in case here it is: smbclient.zip. Keep in mind you will need to install Cygwin on the same machine and at the very least copy the DLL files it needs out of C:\cygwin\bin to the directory with the client.

Hosting Your Own Cloud With Pydio

Last year I was looking for a cloud hosting solution and I decided on Pydio and even wrote up a install guide (http://allandynes.com/2015/08/was-searching-for-a-cloud-provider/). Some time has gone by and I’ve updated those instructions for the latest version of Pydio (6.4.1 currently) and also to be a purely 64-bit install (PHP 7 x64, MySql 5.7 x64, etc). The overall effect on a Server 2012 / IIS 8.5 install is quicker performance and less issues with large data syncs. I’ve also updated it to include information on SMB shares so you can access your existing Windows based file server from your cloud server so you are not duplicating files. Here is the new install guide:

Installing-Pydio-on-Windows-Server-2012-R2-Pure-64-bit-With-Samba

Windows 7 Printer and Spooler Issues 0x00005c3

During my many years troubleshooting computers nothing seems to be as frustrating as printing issues. Granted it has gotten much better but that just means when you do have a issue it’s going to be that much harder to fix. Today I had such a issue. I upgraded the drivers for some network printers ahead of a printer change-out, updating some Lexmark, Konica, and HP Universal print drivers to their respective latest versions. Windows 7 usually is good about grabbing the updated drivers form the server and rarely will anyone see any issues. Well during my testing I had installed and uninstalled the Konica universal drivers multiple times and apparently something got corrupt. When trying to install the new printer I got a error saying Cannot connect to Printer along with a 0x000005b3 code. I did a bunch of research and couldn’t find anything that helped. Some posts said delete your temp files, some said look for *.tmp files within the Windows System32 directory and delete those. Others said try running a Microsoft Fit It for the print spooler (50984 for easy fix and 50979 for full fix resetting everything and deleting all your printers). None of these worked.

Then a post said to check your Windows\INF folder and the setupapi.app.log file for clues. I renamed the file then tried installing the printer again and it created a new log file just with info from the failure. It referenced a bunch of missing files from the Windows\System32\DriverStore\FileRepository directory on the computer. So I though well I can just delete those files and it should download fresh. Unfortunately I couldn’t delete the directory. So now I did a search on how to delete files from the DriverCache and found this article: https://technet.microsoft.com/en-us/library/cc730875.aspx?f=255&MSPPError=-2147217396

Long story short the utility pnputil.exe is used from a administrator command prompt to list all the driver packages cached/installed on your machine. pnputil.exe -e lists all the OEM?? numbers and gives a description of what they are. I ran it and looked through the list and found three different Konica drivers listed. I then ran pnputil.exe -d Oem??.inf for each of the three and it deleted the drivers. I checked the FileRepository directly and the directory in question was gone.

I then tried re-adding the network printer and it worked, connected and downloaded the drivers off the server without issue, and has been working since. So if you are getting the 0x00005c3 error try deleting all printers using the same driver then using the pnputil to delete out an drivers cached and then reinstall. Might save you from re-imaging or reinstalling your OS which many people ended up doing when they couldn’t figure it out.

VB.Net Drag and Drop from Outlook

One thing I have been working on for years now is a database program that among other things has a part that lets you attach files into it. The drag and drop functionality was easy for regular files on the hard drive but a feature request came in to drag and drop files from a outlook attachment. Luckily someone had already figured this out in C# which I converted into VB here: http://www.codeproject.com/Articles/7140/Drag-and-Drop-Attached-File-From-Outlook-and-ab. This was all fine and dandy until I got a request last week to attach the actual email itself, not a attachment. After more searching I found someone that did this using the Outlook interop libraries here: http://www.emoreau.com/Entries/Articles/2008/05/Dropping-a-Outlook-message-on-your-application.aspx. So after a little more playing I updated my drag and drop code to decipher between a normal file, a Outlook attachment, or a Outlook email and act accordingly.

Couple notes: DisplayMessageBox is a custom message box used by my program (replace with MsgBox if needed) and AddTempFileToArray does exactly that, adds any temp files I create or use to a array which I then delete when my program closes (don’t want a bunch of temp files created and left), and SaveButton is enabled only during a edit operation.

In my control where I will accept the drop:

Private Sub MyControlToAcceptTheDrop_DragEnter(ByVal sender As Object, ByVal e As System.Windows.Forms.DragEventArgs) Handles MyControlToAcceptTheDrop.DragEnter
' Make sure that the format is a file drop.
If (e.Data.GetDataPresent(DataFormats.FileDrop)) And (SaveButton.Visible = True) Then
e.Effect = DragDropEffects.Copy
ElseIf (e.Data.GetDataPresent("FileGroupDescriptor")) And (SaveButton.Visible = True) Then
e.Effect = DragDropEffects.Copy
Else
' Do not allow drop.
e.Effect = DragDropEffects.None
End If
End Sub

And my code to handle the drop:

'''

''' Handle File Drops
'''

''' DragEventArgs ''' Path to the actual file or temp file
''' Returns the full path to the file being dropped or to a temp file that contains the file in memory (for use with Outlook or other program drag drops)
Friend Function HandleFileDrops(ByVal e As System.Windows.Forms.DragEventArgs) As String
Try
If e.Data.GetDataPresent(DataFormats.FileDrop) Then
' We have a file so lets pass it to the calling form
Dim Filename As String() = CType(e.Data.GetData(DataFormats.FileDrop), String())
HandleFileDrops = Filename(0)
ElseIf e.Data.GetDataPresent("FileGroupDescriptor") Then
' We have a embedded file. First lets try to get the file name out of memory
Dim theStream As Stream = CType(e.Data.GetData("FileGroupDescriptor"), Stream)
Dim fileGroupDescriptor(512) As Byte
theStream.Read(fileGroupDescriptor, 0, 512)
Dim fileName As System.Text.StringBuilder = New System.Text.StringBuilder("")
Dim i As Integer = 76
While Not (fileGroupDescriptor(i) = 0)
fileName.Append(Convert.ToChar(fileGroupDescriptor(i)))
System.Math.Min(System.Threading.Interlocked.Increment(i), i - 1)
End While
theStream.Close()
' We should have the file name or if its a email the subject line. Create our temp file based on the temp path and this info
Dim myTempFile As String = Path.GetTempPath & fileName.ToString
' Look to see if this is a email message. If so save that temporarily and get the temp file.
If InStr(myTempFile, ".msg") > 0 Then
Dim objOL As New Microsoft.Office.Interop.Outlook.Application
Dim objMI As Microsoft.Office.Interop.Outlook.MailItem
If objOL.ActiveExplorer.Selection.Count > 1 Then
DisplayMessageBox("You can only drag and drop one item at a time into this screen. The first item you selected will be used.", "One Item At A Time", , FormStartPosition.CenterParent)
End If
For Each objMI In objOL.ActiveExplorer.Selection()
objMI.SaveAs(myTempFile)
Exit For
Next
objOL = Nothing
objMI = Nothing
Else
' If its a attachment we need to pull the file itself out of memory
Dim ms As MemoryStream = CType(e.Data.GetData("FileContents", True), MemoryStream)
Dim FileBytes(CInt(ms.Length)) As Byte
' read the raw data into our variable
ms.Position = 0
ms.Read(FileBytes, 0, CInt(ms.Length))
ms.Close()
' save the raw data into our temp file
Dim fs As FileStream = New FileStream(myTempFile, FileMode.OpenOrCreate, FileAccess.Write)
fs.Write(FileBytes, 0, FileBytes.Length)
fs.Close()
End If
' Make sure we have a actual file and also if we do make sure we erase it when done
If File.Exists(myTempFile) Then
' Assign the file name to the add dialog
HandleFileDrops = myTempFile
Call AddTempFileToArray(myTempFile)
Else
HandleFileDrops = String.Empty
End If
Else
Throw New System.Exception("An exception has occurred.")
End If
Catch ex As Exception
DisplayMessageBox("Could not copy file from memory. Please save the file to your hard drive first and then retry your drag and drop.", "Drag and Drop Failed")
HandleFileDrops = String.Empty
End Try

End Function

As you might be able to guess I only look at the first email message if multiple are selected and let the user know that also. I’m sure there is a way to loop through multiple files, Outlook attachments, or Outlook emails but I don’t need that functionality so I didn’t code it (but I did warn in case it happens).