Cornering A Bug

I have a Pro, my trusty old circa 2013 model, somewhat on the old side, running MacOSX Big Sur 11.7.10. It works well enough for everything that I need it for, a basic daily driver for work, most of all.

I spotted a great deal at , a 1 terabyte SSD device for a song and so I got it. Then after I unpacked it and setup on it, so it would serve as a backup for my MacBook I noticed a curious bug. Once I had the drive mounted, it worked well, but when I ejected the drive, sometimes it would give me a error that it couldn’t be mounted. I went off to to search, and came across a bunch of people who were experiencing very similar errors with their SanDisk devices. I found that after I rebooted my MacBook, the problem seemed to go away.

With complicated bugs, often times the trigger is either complicated or multi-step, and for a brief while, the problem with this drive returned. It would come and go, seemingly at random. Sometimes when I plugged it in, it would without issue, and other times it would simply not, and off to , where when I tried to force the mount, it refused with the cryptic error and only clear after a reboot.

I started to really concentrate on all the things I was doing, all the little mindless actions that I do at the start and end of the day, because something was causing this seemingly random bug to appear. One day, last week, I caught myself doing the very thing that triggers this bug!

I have my MacBook Pro, it’s plugged into a standard powered USB 3.0 hub, and then I have a 2TB standard external HDD and the 1TB SDD backup drive plugged into the USB Hub. When I eject the drives at the end of the day, and then unplug them both from the hub shortly afterwards, the drives will both re-mount without an issue. Every single time.

When I eject the drives, and instead pull the USB Hub off the side of the MacBook, after that, the standard HDD will mount without a problem, but the SanDisk SDD drive will always fail to mount with the cryptic error.

I don’t know for certain where this bug actually is, but my educated guess is that it has something to do with USB Bus Enumeration or perhaps some sort of flag that doesn’t get set properly either in my MacBook Pro, the USB Hub, or the SanDisk SDD device.

If you have a SanDisk SDD like I have, and it tosses random mount errors on Big Sur, give this a shot. Eject and then unplug the device, see if that doesn’t clear it up, because it did for me.

How 7-Zip, Hyper-V, and DNS Paralyzed A VOIP Phone System

Today was a tour-de-force in unintended consequences. It started with an old coworker, as a kind of boomerang. They came to work for us, then they moved on, only to come back. That was the premise of this story, the start of it, a coworker boomerang.

The task was really straightforward. De-compress the previously compressed user files related to this particular coworker, so that when they login, they see exactly what they left behind. It was modest, about 36GB worth of data. Looking at everything, the intended target had 365GB of open space, so plenty of room for this. I started with 7-Zip on Windows, opened the archive and extracted it to the drive with all the space. Near the end of the transaction, 7-Zip threw an error, “Out of Disk Space.” and I frowned and scratched my head. 365GB open space, and… this? Turns out, 7-Zip on Windows, at least this copy of it, unpacks the archive to a temporary folder on the temporary resource that Windows assigns, by default this ends up on C: drive. The process was filling an already low-on-capacity primary OS drive. I chased the temporary folder and removed it, correcting the issue. Or so I had thought.

An hour later, out of the apparent blue, around 12:30pm today, all the VOIP desk phones suddenly went “NO SERVICE”. I scrambled, naturally, feeling that rising panic as nothing had changed, there were no alarms, just suddenly total phone failure. I called the VOIP support line, and the official line from support was to reboot my network. A stack of eight fully packed Cisco Catalyst switches, three servers, and a gaggle of networking gear designed to offer at least a dozen vital services – reboot all of that. While talking with support, I opened up a console to my Linux box running on Hyper-V on one of my servers, which is to say, plugged into the very network core itself that I was asked to reboot. I then found my out-of-service desk phone, it’s IP was fine, it was totally functional, I grabbed the SIP password, logged into the phone, went to where it lists the VOIP endpoint for our phone carrier, and then asked mtr to show me the packet flow across the network, from my humble little wooden box of an office to the VOIP endpoint. The utility was clear, it was fine. No issues. 500 and counting packets all arriving promptly, no flaws, no errors, and NO SERVICE.

So I was growing more vexed with support, really unwilling to reboot the entirety of my network core when mtr was just merrily popping packets directly to the correct VOIP endpoint deep inside the carriers network. My traffic could get to where it had to go, the phones were NO SERVICE still. Support was flat-footed. I stopped myself, because I could feel the rage build, my old companion, the anger that comes when people aren’t listening to what I am trying to tell them. I stopped. It was not going anywhere and I promised myself that I would fight this anger, tooth and claw to the best of my ability. So I simply calmly asked for the ticket number on their side, and thanked them for their time and hung up my cell phone. I obviously muttered some choice phrases in a small voice, but otherwise I was very proud of myself. I derailed what could have become a very ugly scene.

Everything works. I am not going to reboot the core. The phones simply say NO SERVICE. Then other reports rolled in, network faults, adjacent but not the same, Wifi failures in Houston Texas, hmmm. What does Wifi out in Houston have to do with dud phones in Kalamazoo?

I had this sinking feeling, my gut screamed at me, something about the PDC, Wifi, and the Phones were all touching something common that had failed, but had failed silently. I chuckled to myself, the old IT chestnut occurred to me, “It’s always DNS.” and so, in respect to that, I opened the Hyper-V management window on the PDC and looked for my twin OpenDNS Resolvers, they are VM’s that run quietly, flawlessly, for years on years without a peep deep within Hyper-V. There it was, right there, right in front of me. The two resolver VM’s and just to the right of their names, the quaint little status indicator from Hyper-V. “PAUSED.”

The moment I saw that, I yelled out “PAUSED” and “NO SERVICE” and screamed. Right click on both VM’s, click Resume, and Hyper-V gleefully, in a heartbeat, resumed both little VM’s and just like that, another reboot to the VOIP phone and bleep-bloop-blunk, the phone was functional and just fine.

It is always DNS. I have three resolvers, the two resolvers were on the same host and the host had a wee panic and Hyper-V silently just paused everything, and then after a short while of cooking, the phones and Wifi, which also uses those resolvers, all went kaput all in one happy bunch.

Obviously the answer is to round-robin the resolvers, the primary on the PDC, then one resolver running in VMWare nearby, and then the secondary on the PDC. A sandwich right down the middle. I both thanked my past self and kicked my past self, for having the wits to set up a third resolver, which was then for a short while, the only resolver there was, except for choice parts of my network.

So, it ended happily, alls well that ends well. The next step is to spread this round-robin resolver correction throughout my network, to help avoid this from ever happening again. But then I laughed as I considered the gamut of what had transpired. 7-Zip, well meaning and purely accidentally caused an unintended disk space alert, Hyper-V silently and studiously paused its charges, and the network kind of rolled on over the speed-bumps, and at the end, proved again, “It’s always DNS.”

Anti-Spam Whitelist Email Strategy with the Plus Character

A friend of mine on the group that I am a part of was lamenting how one of his email addresses that only exists for billing needs ended up on a spam-list. That started me thinking about how I might address this, in a really needlessly geeky way.

A lot of email providers allow folks to use the + (Plus) construction when it comes to email addressing. This construction will deliver to the primary mailbox, which is before the + character, and then discard any tag after it, unless you want to use it for categorization in email rules. Which is exactly where I was going with this entire idea.

So start with an email, probably best to pick a provider that is free, say we pick gmail.com, so a new email address, lets call it John Doe, the username is jdoe@gmail.com. Obviously, just an example.

We set up rules, in the rule we throw any email directly addressed to jdoe@gmail.com right in the trash. Then we set up uuid’s for each service using uuidgen, which is an 128-bit value guaranteed to be unique in space and time. So we can set up a vendor list, like this:

  • Google – E11849AF-60E5-4008-A23F-2DCDCC970DF4
  • Microsoft – A268F0C4-C17D-4BC1-BA46-E6E4E7DC856A
  • Apple – 1969FF7F-49DA-487C-9692-0AECCFC62E58

So then, when we enter in our email addresses in these unique services, we can just go with these:

  • Google – jdoe+E11849AF-60E5-4008-A23F-2DCDCC970DF4@gmail.com
  • Microsoft – jdoe+A268F0C4-C17D-4BC1-BA46-E6E4E7DC856A@gmail.com
  • Apple – jdoe+1969FF7F-49DA-487C-9692-0AECCFC62E58@gmail.com

Then right above the email processing rule throwing all the email to jdoe@gmail.com in the trash, we define each of these To: addresses as authentic, and then we can do stuff with the emails in the rules, like send to a folder for example, or mark it, throw a sound, webhook, whatever.

If any vendor gets clever, spots the + construction and strips it, they get nothing, as we are throwing all the messages addressed to jdoe@gmail.com and jdoe+@gmail.com right in the trash without even seeing them. If emails come in from our contact services and they are really meant to be, they will arrive.

What if we start getting spam with an address? Re-roll the uuidgen, then change your email address to the new one. So now if Google sells our address to a spammer, we just create a new uuidgen, like 49F8179E-F171-4783-9246-8D3C46532575, and then go to google and change our email address to jdoe+49F8179E-F171-4783-9246-8D3C46532575@gmail.com. Then the rule where any email that isn’t matching the right uuids gets all thrown in the trash.

This way, you’ll never get another drop of spam ever going forward. If a site sells your address, just change your email address, and burn the old uuid.

It’s one way to address spam, by a scorched-earth policy. We can’t have nice things, so we have to do it this way.

Subdirectories, Apache, and ActivityPub

I have this terrible feeling that I may have fixed my ActivityPub issue. I was able to find this blog from my Mastodon account and follow it. If it continues to function, that’ll be the next unexpectedly pleasant surprise.

Turns out, my blog lives in a subdirectory. https://www.windchilde.com/bluedepth. The plugin flavor text, at the shallow end of the pool makes absolutely no mention of blogs living in subdirectories. It took a Github issue that ended up being one of maybe six that I ended up checking out. Here’s the page that won the day:

https://github.com/Automattic/wordpress-activitypub/issues/538

And the most important part was the blind axe-throw at .htaccess from the root of my hosting account, to cajole Apache into behaving differently than was intended. Here’s the missing bit:

“RedirectMatch “^/.well-known/(webfinger|nodeinfo|x-nodeinfo2)(.*)$” /blog/.well-known/$1$2″ – without the quotes around it, and replacing “blog” with your subdirectory.

After that was done and saved, went back to Mastodon.social, search, and put in the term @admin and poof! There it is! It found it, I could follow it. Suddenly feeling very strange because I had given up on ActivityPub completely. Apparently it works. Huh.

Well, I say that it works. This post might show up on my Home feed in Mastodon.social, if all the trained hamsters aren’t smoking meth, I guess. LOL. Only one way to find out… and hit that publish button!

ActivityPub Lulz

Looks like adding ActivityPub plugin to this updated WordPress install isn’t working. Site Health says everything is absolutely fine, and that’s the end of the Troubleshooting guidance from the plugin.

It didn’t work before, it’s not working now, and we’re really not surprised in the least. I seriously doubt the plugin will ever work. It might work on any other host, but not iPage. Without being able to find @admin from Mastodon, there can’t possibly be any followers, and since the plugin doesn’t actually *do* anything by itself, this blog will never be visible from the Fediverse.

I’d be shocked if it ever worked. So, this blog will likely go back to being ignored and derelict. I seriously doubt anyone reads this trash, so it’s for the best.

Noises Off! LOL!

ActivityPub and the Fediverse

I just updated my blog, finally. It took forever, because the automatic-by-the-link update method no longer works, so I had to download WordPress fresh, put it somewhere handy, and then follow the upgrade instructions I found on the Automattic site for this sort of thing. Once I had all the files updated, I refreshed my blog and it asked if it was okay to update the database, which took only moments.

Then once that was accomplished, the next step was to update PHP on my silly host, they are not helpful at all, and now that I have updated my WordPress manually, I don’t know if I ever will need their help again.

After all of that, I installed the ActivityPub plugin. It says it is functioning, but I have no idea if it really is or not, I can’t seem to get it to come up on Mastodon.Social, so perhaps a new post will trip some trigger.

The notion of writing in my blog, for long form pieces is something that might be really useful again, now that I am no longer on Facebook and this promise of expanding my social reach using ActivityPub certainly is attractive. I suppose only time will tell.

Kludgey

This post was written on Mastodon so it has an informal writing style with jargon and a manner for a specific audience. All the spelling and grammatical errors are intended.


I love creating my own problems and then finding a rabbit hole and chasing it until I have a geek solution that is likely foolish. If I load too many tabs on my Macbook, it gets sluggish. So, can I start xQuartz? Sure! Update however…

Okay, that done, can I SSH with the -X flag to my little Raspberry Pi? Yes! Sluggish. Wah.

How about to my “Security” laptop, running Linux? Yes. Sluggish still.

Google Search, find x2go, install it. MUCH BETTER.

So I’m using x2go, running Firefox-esr and connected to my not-work-tabs, including this one. Not seamless, but it works acceptably well enough.

Sitting here, marveling at all this exceptionally complicated computing technology before me, everything has “multiple cores” yet you really couldn’t tell. So instead of running everything from one single computer, we’ve got serious work stuff on one, then a remote desktop window to another running “fluffy stuff”, and then playing Spotify from my !@#$ iPhone. HAHAHAHAHAHA.

Shitty apps, each written by devs that believe that their app is the “King Of The Realm” and you can malloc() forever without having to fret over anything at all. Leaks? Who cares! Look how pretty it is! So, multiple computers, multiple OSes, failures aren’t less, but they are spread out so they don’t block real work quite as badly.

Of course, there is also I/O Blocking to contend with. When the filesystem is doing anything, everything stops. Because I/O is super smexy.

So we contend with shitty development choices by simply throwing entire chunks of technology at the problem. Two laptops, a Raspberry Pi, an iPhone, and an iPad. Each device is good at individual things, but no, we can’t do everything on one single device. Watch that device just chug right to the fucking ground. Ah well. The modern response is “throw a hypervisor on it” and that, wow, what a great way to make an even bigger mess of things.

Bullshit hypervisors make for hilarious blown-out-afternoons. So, Windows 10 on an HP Elitebook laptop, install Hyper-V from the OS, and the Radeon display driver commits hairy suicide. Not only does the driver break, but it cannot be “upgraded” or “fixed”, the only thing you can do, is remove HyperV and… poof, uh, there was a problem? No! No problem! So, you shrug and chuckle and look at the icon for VirtualBox. Yeah, hey buddy…

There are some situations where I start thinking that I should buy a cheap $200 Chromebook just for some things. More technology. SMH. Of course.

Two days ago I remembered the glory-promise of X-Windows and SSH tunnels, with the Display being sent elsewhere. Oh my god, the promise of that… so glittering. So… disappointing.

Oh it works. But it’s like watching slugs have a romantic dinner. Maybe I should just read a book while you request that website, hmmm?

Obviously you turn to Google, the eminent sage and eternal junkie for answers. Ah yes, X-Windows over SSH is a ping/pong nightmare, half the traffic is consumed by just making sure that all the lower layers are functioning properly, constantly. Fine. But then you spot things like x2go, give that a shot, eh… it’s somewhat better.

In the end, the promise bends to tools you already have. Like TeamViewer connected to Windows 10 on a different laptop.

Heh, assuming TeamViewer stays functional that is.

Technology is bittersweet. We have such command of so many wonders. I can’t shake the feeling that it’s all an immense house-of-cards. I suppose I’ve seen too much, I know too much, “I’ve seen things you people wouldn’t believe” kind of running through my head. Like looking at Layer 1 connections secured by… chewed bubble gum.

And all the various cheats and hacks, because you naturally want something, but you can’t get it because the people who provide the thing, won’t provide the tools for the extra things you want. They aren’t going to write the code, their codebase is secret, you can’t submit code yourself, and so you just sit there, google searching and finding kludgy-as-fuck solutions to your headaches.

Yeah, that’s fine, be a prat. I’ve got a python script that scrapes your shit and does what I wanted to do.

Case in point, Signal. I love Signal. I will always love Signal. But I want to search on metadata within the Signal app. No. Like the soup nazi, no metadata for you! Only search on stuff in primary stream. Fuuuuuuuu.

So I have a group chat, it’s all my friends, in Signal, and we use it as a blazing-bright thread, it connects us all, geographically spread throughout the United States. It made the COVID-19 Pandemic less lonely. It was a community of dear friends and we could be together without risk.

So, I journal, have an app for that, but obviously Signal doesn’t work with the journal app, so I can’t just hoover all the Signal content into the journaling software. Sometimes I forget to review everything we said in Signal, so the date-of-chat just slides off primary display. You could scroll, but wouldn’t it be nice to search on metadata? Like take me to the first thing shared on Sep 01, 2021? That would be nifty! NO. NO SOUP FOR YOU.

So, no metadata searching. Fine. So, enter the raw kludgy “fuck it, this is also a solution, damn you all” solution. Can’t search on metadata, but just on raw data, so, lets add the data markers we want to the stream! At 6am, write the date into the stream, every day. Then you can use the tools in the app to search on what was shared, and since the metadata you want is “shared”, now you can search on it! Well, okay! “Sep 01, 2021” look! YAY! That’s what I wanted!

Obviously this creates a “Forking House Of Mirrors”… one bullshit kludgy solution leads to a new problem. I don’t want to wake up at 6am to put the date into Signal stream. OK. Lets automate that. Enter Signal-CLI. shakes head fine. So, lets try to connect to the service, that was a hard climb. Okay, now it’s as group, what groups are there? No groups. What? No. Send something to someone, then ask again. Okay. <<send>> how about now? OH YES, THIS GROUP!? You need a special hex code for this.

If you have this hex code, you’d think you could use that without having to ask going forward. No. New install? You can’t just simply use what you know to peek around the corner, no. You need to run around Robin Hoods Barn all over again, and now you can use it! HUZZAH. FUUUUUUUUUUU.

So, finally, we can send signal data from the CLI. Next, lets figure out the date commands picky-picky formatting rules. How to get Sep 01, 2021?

We’ve got that! YAY! Okay, so lets write a Bash script! Get the date, and at 6am write it out to the Signal group. Write script, change mode on script so it can execute, plumb the foggy memories you have of crontab, and boom. Failure.

FUUUUUUUUUUU

Ah yes, cardinal sin, I didn’t explicitly declare the specific paths to signal-cli, echo, mv, fuck, any command at all. Call the script yourself, works, cron calls? Lost. Fixup. Dive into vim. Find your cheatsheet. Gah.

Finally, good god watch it work. 6am every day, a machine you “rescued from the landfill” with some half-forgotten linux distro you can’t remember is actually working and that’s fine. Now, when it’s Sep 03, 2021, you can search on Sep 01, 2021, to get back and manually journal what you remember telling people, because there it is. Click-drag.

All because metadata isn’t searchable. I got what I wanted. Everyone can benefit from it too. But it is complete mess.

This is why entire afternoons are incinerated on the pyre of “Fuck, I wanted XYZ, but the devs don’t speak English, their angel investors aren’t interested, and nobody but me would ever want this feature… so… fuuuuuuuuuuu”

I suppose I could attempt to ask for whatever it is I think would be good, but devs live on the moon, or as much as would be useful, they do. So no. We don’t tell devs anything. We just muck about, finding fragments on GitHub, trying not to get sick that Microsoft owns them now.

So you find gists, you find forked projects, you find python code fragments. The dependencies aren’t circular-misadventures-into-the-fog, you try to remember basic linux stuff because you haven’t had to screw around with any of it for decades and crontab went off to the same Elysium Fields that Trigonometry went off to…

Google Fu. Another worrisome “house of cards” right there too, but lets not look too closely at it, lest it collapse. Or sell our identity to Belorussians.

It doesn’t take much at all. Fragile houses of cards built on other fragile houses of cards. People mobbing on top, like hapless Eloi sitting down at the picnic tables and never having a single bright shiny thought in their pretty little heads because food is always right there, on the table, same time every day. Meanwhile, underneath, the Morlocks are banging on pipes, and every once in a while grabbing an Eloi for a snack.

That’s the Internet. Humanity on top of the Internet. The rot in Layer 8.

And all you really do is shrug. You hope for a better world. Every once in a really long while you stumble blindly over something truly elegant. It’s like tripping over Rivendell and spotting an Elf walking along a curated beautiful path of perfectly carved scrollwork.

And it’s only momentary. The pile of constantly shifting wreckage we call the modern world continues to shudder and throb. It all works, and you marvel that these people manage to continue to live in all of this… wreckage.

VAR Blues

I had to step away from the VAR I was using at work because of a recent change they had instituted with my business account. For years, I had enjoyed a classic relationship of having a single VAR Account Executive assigned to my account, where the AE would learn from me and get to know me, and I would get to know them as well. It was a very successful working relationship, and had been the way of things for six years. Around two months ago, the company made a change. They moved my business account from the structure that I was familiar with over to a team-based structure, and billed the benefits to include “There will always be an AE to work on your account” as a value-added proposition. I was worried that the change would instead eliminate the engagement, the learning, and the developing relationship between customer and reseller.

This new structure included a single shared email address that many people had access to, the AE’s assigned to the “Pod” and the “Pod Manager” who also kept a view on the shared mailbox. I was supposed to send every correspondence to this shared address. At first, I enjoyed the value proposition that there would always be someone to get my messages and to execute my requests. Although, to be brutally honest, access to my Account Executive was never really a problem, so this value proposition was actually a “solution in search of a problem” that I didn’t have. It wasn’t until much later, in the retrospective analysis, that I came to realize this as more significant than I considered it at first.

It was after this, when the rest of the feature set for this new structure started to appear. I’m certain that the VAR thought that all of these things were only enhancing value for customers, but really every step just led me further away from where I was most comfortable. I wasn’t able to “get to know” my team of Account Executives, they remain faceless, voiceless text in email. This lack of humanity was at first not considered to be an issue, but later on became significantly problematic. The disconnection accelerated as we progressed. I was no longer handing work to an Account Executive, asking them for advice and tips, and there was a significant amount of value that I was suddenly unable to access. I had come into the arrangement with a habit of asking my Account Executive to send me quotes on various items, and they would seek the best fitting item that suited my preferences and hand me a quote for the recommended items that best fit my needs and, during pandemic, had a better chance of being in stock. This habit was broken by the new way of doing things. I was no longer able to reach out to an Account Executive to get advice, to have their vantage point much closer to the manufacturers and distributors that we all were using to acquire technology, now I was supposed to simply go on the VAR website, find what I wanted, do my own shopping, and then assemble my own quotes. This feeling of being cut loose became pervasive because it was just another touted feature, considered by the VAR to be part of the “Value Added”, and quickly included not only writing my own quotes, but submitting my own orders as well.

The loss of engagement, the anonymity of the Account Executives involved, and how I was supposed to move all of my previous activities to self-directed work, ostensibly leveraged on the VAR website, all touted as “value added” components were actually just the opposite for me. It wasn’t until I started actually living in this new environment, doing my tasks this new way, that I realized just how much I had missed the old way that I used to do things. The value proposition was always above board, nobody was intentionally being manipulative or malicious, but the result was cold, impersonal, and made me feel like there was an erosion of all the value that at one point was part of my “value added” experience with my VAR.

Whenever there is a change, items can be lost in translation, they can get missed, I do not fault anyone for missing say one or two small things as the customer and the Account Executive in the VAR start to grow together and establish a working relationship together. I didn’t want to, at the time, hold people’s feet to the fire, but that’s exactly what I ended up having to do. I maintain a strict three-strikes policy when it comes to faults, if it’s awful, and you did it three times, that means that it isn’t a mistake, it isn’t overlooking something, it’s part of the design.

The first fault was completely missing the deadline on renewal of security software that my company depends upon to protect us all online. Thankfully, the manufacturer has a very gracious fifteen day grace period, where deadlines are much softer than how they actually sound. The fault resolved, and we moved forward. The second fault came shortly after the first one, and again, the same manufacturer. Missing the renewal of contractual agreements that enable me as a customer to approach the manufacturer’s technical support center if I have any questions or problems. It was addressed and we landed on our feet, but again, we had to sag backwards into the fifteen-day grace period. The third strike was one of tragic poor communication, and one of the most egregious failures I’ve ever witnessed. This failure also coincided with a new Account Executive team member whom I had never communicated with before.

The lack of experience and knowledge on both sides of the divide, again, became a problem that really got in the way. This new Account Executive asked me over several email exchanges questions that were too vague to answer because there wasn’t any included detail. IT is a detail-centric category. We thrive on details, we need exact details, like numbers, or topics, some way to clearly identify what it is that we are talking about. It doesn’t really work when people try to use vague communication styles packed with pronouns and references to unknown objects. Exchange after exchange in this manner became tedious and incredibly tiresome. After several iterations, where I had also started carbon-copying the Pod Manager, did the truth of the situation reveal itself. Once I learned what the object of the conversation was, I tracked it and realized that the subject work should have been completed months before when they had already invoiced my company for the work completed, invoiced and paid.

That was the last straw, the VAR relationship had a tragic and lethal attack right on that spot, right at that time. I began to pursue a kind of “re-entry to the VAR marketplace”, essentially shopping for a new VAR. I found one, chatted them up, had several fantastic meetings and the new VAR has more energy than I’ve seen from the previous one, more professionalism, and more effectiveness. Furthermore, I was also clear with the old VAR, telling them that it was unconscionable how things had unraveled between us, including the “Pod Manager” who never even once attempted to intervene. It was like complaining at a brick wall, for all that I got out of the subsequent correspondences.

The way I was treated was more educational than bothersome. It was a lesson for how important my companies account was to the old VAR, that during the COVID-19 Pandemic, our purchasing slowed because the supply channels also slowed. We wanted technology that was on extensive backorder, and so as our purchasing slowed and stopped, our value to the old VAR ebbed away. The group arrangement was a lesson in and of itself, we were too small, too insignificant to assign to a singular Account Executive, and so, we were effectively downsized as a customer.

We were expected to do all our own work, be our own VAR, as it were because we simply weren’t buying enough to be relevant to our previous VAR. This in itself carries a rather embarrassing knock-on side effect because we had ordered a particular kind of technology from a particular manufacturer and we had eleven items on extreme backorder with the VAR. The old VAR never valued our account, and this was proven out to us by the later revelation that the eleven items on extreme backorder actually slipped into “End Of Life” from the manufacturer. The VAR couldn’t be bothered to re-evaluate the old Open Orders unprompted, discover the EOL surprise themselves, and try saving face by explaining to us what had happened and offering alternatives. What had happened instead, was that the customer had started conversing with a new VAR, discovered the EOL condition, that highlighted just how little the prior VAR cared.

It didn’t matter what the old VAR even wanted to attempt in recovery efforts for the now fully dead business relationship because the single thing that they bring, their “Value Added Reseller” nature, was proven to be totally absentee. We didn’t buy technology for lack of funds, we didn’t buy technology because the people meant to handle the reselling never noticed that what they had already sold ceased to be for sale by the manufacturer!

So I walked away. I moved many orders from the old VAR, spec’ed them out with the new VAR, and actually ended up solving nearly all the seriously backordered gaps in our purchasing stream in one singular afternoon. I sometimes wonder, idly so, if the old VAR thinks about the suddenly cancelled orders, where we were waiting since April with extreme backorders, and then interest fades. Do they even care, do they even notice? They didn’t care enough to look at any old open orders, to even see that the item that we were waiting for shipping on was never ever going to ship because the manufacturer simply stopped selling it. Not having the attention to detail on historical items makes it not really any surprise that they kept on fumbling until the customer simply walked away.

I think that the critical lesson for me in all this drama with the two VARs has been the hidden value that engagement had throughout the entire experience. I never really had a visceral feeling for just how important the engagement was between myself and my VAR Account Executive until it was eroded almost completely. Once engagement disappeared, it was a game-changer for me because it illustrated just how important VAR Account Executives are in the process, how much I had come to rely on them. The VAR Account Executive sits in a very high place, able to see things that customers cannot. Their fingers on the pulse of international transport, distribution, and delivery. I only wish that my prior VAR had not allowed six years of solid, dependable positive experiences go down the drain like it did. I am far happier with my new VAR. With the right engagement between customer and Account Executive, I have already spent $25,000 with the new VAR! These dollar values are still small potatoes in comparison to other customers, but $25,000 is certainly more profit for the new VAR than for the old one who is just sitting on a pile of cancelled orders.

Meraki Z1 & Cisco 2801 Link Negotiation Gremlin

Today at work I ran into a really long-standing issue that we’ve had in one of our company branches. This branch uses an EOL/EOS Meraki Z1 Teleworker Gateway and also uses a hilariously EOL/EOS Cisco 2801 Integrated Services Router.

The setup is very straightforward, on the Internet side of the Teleworker gateway is a Comcast cable modem, and it’s only capable of 60mbit downlink and 10mbit uplink for maximum speed. We rebooted everything, re-tested from the cable modem and then to the desktop itself, and the speed from the cable modem was just as we expected, 60/10, but the speed from the desktop was 4/6!

I had rebooted everything. The cable modem, the Meraki Z1 Gateway, the Cisco 2801 ISR, and the Cisco 3560 Catalyst switch. Even the Cisco IP Phone got a reset! The speed gremlin held out, 4/6. So while working with some staff in the branch, I just happened to mouse-over the graphic on the Meraki Dashboard for this device and spotted the gremlin. The mouse-over tip for LAN1, where the Cat5 cable goes from the Meraki Z1 to the Cisco 2801 showed 100mbit/half-duplex. I checked into the terminal on the 2801 and verified that the port was fixed at 100mbit/full-duplex! So, I opened the Meraki Z1 device Ethernet configuration page, found LAN1, and changed it from “Auto” to 100mbit/full-duplex.

Forcing the speed and duplex settings resolved all the problems right out to the Desktop! Hooray! And what I learned from this is that Meraki Z1 Teleworker Gateways cannot successfully auto-negotiate link speed and duplex with a Cisco 2801 Router. So if you have unexplained crappy network performance, always make sure that link speed and duplex match what you think they should. Sometimes “Auto” isn’t.

Photo Credit: Gremlin Grotesque, Winchelsea church
cc-by-sa/2.0 - © Julian P Guffogg - geograph.org.uk/p/3334405

Ugly White People, Wearing Masks, and Leaving Facebook

So earlier today, after leaving our local megamart, which in this case is Meijer I was beset by wave after wave of ugly white people not wearing masks. I just cannot stand it, the absolute gall to put the public health at risk all because you want to be a dick about it. It is just beyond acceptable, even in our broken world, so I wrote a Facebook Post. I called them for what they are. Ugly White Pig Fuckers.

The Facebook AI flagged it as “Hate Speech” and so, since I have a long track record of calling out Russians for their shenanigans along with I’m sure other infractions that I have long since forgotten, I have been put in a time-out corner for three days on Facebook.

Almost everyone that I care about is on a shared Signal group, it’s a virtual pub where all my loved ones are also there and I can vent, and listen to my loved ones vent, and we can laugh and share things and because Signal is end-to-end encrypted, there is nobody there to tell me what I can or cannot say.

Very much like this blog too. I always mean to write more here on the blog, and this time-out from Facebook for 3 days is actually not a punishment but rather an invitation I think, to fully abandon the platform. The toxic people, the toxic stories, the endless and sensationalized bottom-of-the-barrel scrape that the wall has become. If I want to visit a wreched den of scum and villainy, at least Reddit doesn’t pretend that it is anything else than just another cultural latrine. Facebook is just a lemon-scented cultural latrine.

I pay for this blog and the service, so I can say whatever I please here without an obnoxious censorship AI locking my account out. Plus, it’s like TV, if you don’t like what I write on this blog, you are very much invited to forget all about it. Just don’t point your browser here, I will not be offended.

So instead of sharing things on Facebook, I’ll share them on this blog. The activity will pick up, maybe if I’m very lucky there will be a new community like Imzy, or perhaps something like LiveJournal before the filthy Russians got their grasping little fingers all over it. Everyone who reads the blog should know, I’m left AF, and while I am not Antifa, I am Antifa sympathetic, especially with the notion that anyone of good standing and solid heart will not hesitate to punch Nazi scum in the face.

So don’t look for me on Facebook. Look for me here. To Hell with Facebook.

Also… WEAR YOUR !@#$ MASKS IN PUBLIC!