Many of my traditional blog post live on this site, but a great majority of my social-style posts can be found on my much-busier microbloging site at updates.passthejoe.net. It's busier because my BlogPoster "microblogging" script generates short, Twitter-style posts from the Linux or Windows (or anywhere you can run Ruby with too many Gems) command line, uploads them to the web server and send them out on my Twitter and Mastodon feeds.
I used to post to this blog via scripts and Unix/Linux utilities (curl and Unison) that helped me mirror the files locally and on the server. Since this site recently moved hosts, none of that is set up. I'm just using SFTP and SSH to write posts and manage the site.
Disqus comments are not live just yet because I'm not sure about what I'm going to do for the domain on this site. I'll probably restore the old domain at first just to have some continuity, but for now I like using the "free" domain from this site's new host, NearlyFreeSpeech.net.
I'm not trying to make money with this blog, and it hasn't happened despite my lack of interest in same. I don't have Google Analytics (or Piwik) counting the traffic. Every once in a while I look at the AWstats data from my web host, but not too often.
I write about the things I'm doing. It's mostly about technology, though for the longest time I've meant to write about other things.
If you want to read this site, it's for you. Otherwise, it's just for me.
I am using the unison
in Ubuntu 14.04 (Unison 2.40) in my Windows Subsystem for Linux-supplied Ubuntu 16.04 (which updated the package to Unison 2.48) because my server is running Unison 2.40, and I forgot that an apt upgrade
will replace the .deb
I downloaded from the 14.04 repository with whatever is in 16.04.
When I tried to do a unison
sync, I got an error.
How do you put a package "on hold" in Ubuntu? It's easy.
First I removed the "new" unison:
$ sudo apt remove unison
Then I installed my "old" one (which I had previously downloaded from the Ubuntu archive):
$ sudo dpkg -i unison_2.40.102-2ubuntu1_amd64.deb
Now I put the package "on hold":
$ sudo apt-mark hold unison
That's it.
Here is the output now for sudo apt update
:
$ sudo apt upgrade
[sudo] password for steven:
Reading package lists... Done
[sudo] password for steven:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
unison
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
After a Windows 10 update hosed my laptop and took the Ubuntu/Bash command line (aka the Windows Subsystem for Linux) and all of my scripts along with it, I restored Windows 10 from the laptop's backup partition, and activated the Windows Subsystem for Linux (aka WSL).
This time around I got Ubuntu 16.04 rather than 14.04, which is overall better because there are some really, really old packages in 14.04, including a super-old nodejs
. Unfortunately, the old unison
in 14.04 matched what is on my server (and unison
versions across computers must match, or they don't work).
Luckily I was able to download a 14.04 package from the Ubuntu archive and install it in the 16.04-powered WSL. I restored my scripts, including one I made that is very WSL-specific: It takes all of the files in a Windows directory (usually images, sometimes text documents, but it could be anything), copies them into a working directory in the WSL and uses chmod
to change their permissions to 644. That way I can download images while in the web browsers of the Windows world, create text files, working on all of them with Windows tools, and then transfer those files into the Linux side, where I can sync them to the server's filesystem with unison
.
Aside: It's not impossible to get a Unix-style ssh
program that works from the Windows command line, but it's not at all easy, either. That makes the Windows version of Unison less than useful for working with remote servers.
Now I have scripts in the WSL to:
unison
to sync files and then reindex the blog via Ode's Indexettecurl
to bring the html down to the laptop, then copying it into the local Ode filesystem, and then syncing with the server via unison
.I have a feeling I've written about most of these scripts before, and if/when I find those entries, I will link them here. If not (or if there have been updates), I will write them up in the near future.
Why Unison? Unison is a file-synchronization tool. While files can be synced from one system to another with rsync
, which I use for backups, the situation with this Ode blog is different. Anthor way to synchronize two filesystem is to use git
, the version-control tool.
What unison
enables me to do is make changes locally, or on the server, and then reconcile those changes across both systems. So if I write or edit a post on my local filesystem, or make any kind of change on the server, running unison
ensures that I have the latest files (and versions of files) on both filesystems. If I used rsync, making changes on the server but running rsync
on the client wouldn't work. Git
would be great, except that it only reconciles changes in the filesystem that have been checked in. Changes on the server are generally not checked in, and even if I scripted that on the server, Ode (through its Indexette and EditEdit addins) itself makes changes to the filesystem and doesn't check them in. So git
wouldn't work.
I came up with unison
because it's the easiest. Another alternative csync2
looks a lot harder to figure out. But I do recommend csync2
if you're doing something heavy-duty with more than 2 servers.
When I started looking for this kind of tool, I knew what I needed was a kind of Dropbox for servers. I'm sure there are people who have hacked Dropbox
to work on a non-GUI server. Actually that would be a pretty good solution.
The difference with unison
is that you have to "consciously" run it to sync the two filesystems. You could run it as a cron job, or somehow set it up as a daemon (which might be how Dropbox works), but for the purposes of this particular blog, syncing when needed works fine.
Using the WSL has provided me the opportunity for the first time in quite a while, to set up unison
. It's a great thing to run unison -batch
and have the entire blog filesystem copied to an empty directory on my laptop in about a minute. (And then any changes I make on either laptop or server can be synced with another unison -batch
, or just unison
for a more interactive session. Plus, never underestimate software you can install yourself, on your own computers, and use as you wish. I pay for my shared-hosting service, but otherwise I run whatever software I wish without paying any monthly fees for any of it.
Are there other ways to keep two or more filesystems in sync? I'd sure like to know if there were.
I've always been able to get around in Vim, and before that vi. But it hasn't been my primary editor (except in college, where it was my only editor).
In my Linux systems over the last many years, I've gravitated toward Geany and Gedit, mostly using Geany, and using the terrific Notepad++ on Windows.
Now that I am using the Windows Subsystem for Linux (aka Bash command line supplied by Ubuntu), I have the full range of editors available in the Linux console. For whatever reason or reasons, I'm not an emacs person, and I'm not afraid of modal editing, so Vim it is.
This gives me the opportunity to really learn Vim. Already I'm figuring out things in Vim's command mode
, like w
taking you from word to word and stopping on the first letter of each word, with e
doing the same except stopping on the last letter.
Typing gg
in command mode
gets you to the top of a file, and G
(and also L
) gets you to the top of the final line. G$
gets you to the end of the final line.
x
deletes a single character, dw
deletes a word, dd
deletes an entire line and d$
deletes from the cursor to the end of the line.
It's nothing like a "standard" GUI editor, but a lot of it falls right under the fingers. While I have used an adm3a terminal, it's been long enough that I didn't know the reason for using the esc
key to change from insert to command mode was the placement of the esc
key on the adm3a -- where the "modern" tab would be.
To make it easier to change modes, I don't want to remap tab
as esc
but could try remapping caps lock
as esc
, or using ctrl-[
, ctrl-c
or alt-space
as esc
alternatives. Thus far it doesn't look like remapping caps-lock
in the WSL is all that easy.
I ran Fedora from F18 through 25 and never had a problem getting my laptop to boot and run. But in my first month as a Windows 10 user, an upgrade has already hosed my laptop, causing me to restore it to its original state and reinstall my applications.
I'm not sure what the update was supposed to do, only that it was big and would require a lot of time and a number of reboots.
After a bunch of those reboots and a lot of time, I was left with a black screen and a cursor. That's it.
I could ctrl-alt-del and get a prompt to shut down, but I couldn't do anything else. I think the updates "broke" the video driver.
So since I only had a month "invested" in the OS, I could have wiped the entire thing and put Fedora on the laptop. But I decided to give Windows 10 another chance. I liked having the Ubuntu command line, even though it was the ancient 14.04 instead of 16.04. And I had my blog set up to deploy from that Bash command line.
I opted to reinstall the system and keep my user files, which is one of the options available on this HP Envy laptop. I assume it's the same (or nearly so) on most PCs. There is a "restore" partition that contains a copy of the original OS files, and that is what is used to reinstall the system software.
That operation took a long time, but at the end of it I had a working Windows 10 laptop once again. All of my user files were intact. But as promised, my applications were all gone. I did get a handy HTML list of them, mostly with links to the project web sites. However, I did have the install files for all of them in my Downloads file, and all I had to do was reinstall.
I did lose lots of configuration files.
I still have Vim and Gvim WITH configuration files because I elected to use the binaries from my Downloads file and not "install" them the usual Windows way. So when my laptop came back, the only application icons on my desktop were vim
and gvim
.
In a more grim note, the Windows Subsystem for Linux, aka the Ubuntu command line, aka the Bash command line, is NOT in any user account, nor are the files I created in Bash. That means when I did the reinstall I lost the WSL and everything in it. Pro tip: Back up your WSL files!!
I can re-create what I did in the WSL, though I won't be happy about it. And I have no idea if the laptop now has the update that broke it yesterday, or if it's coming down the pike in the days or weeks ahead. I'm certainly not going to go to the Windows Update screen and click anything that reads "update now," or whatever it says.
I have heard about this black screen issue here and there, but it doesn't seem to be widespread enough to cause any kind of massive panic. And while I'm sure there is some slick way to fix what was broken, I couldn't figure that out, and doing the restore (while keeping my user files) was the quickest, easiest way to get going.
And to elaborate on what I say at the top, if you "keep your nose clean" in Linux, meaning not try to use proprietary video drivers or do anything stupid with dodgy packages, it's pretty hard to unknowingly kill a working Linux installation. I thought the same was true for Windows, but now I know otherwise.
Update: I reinstalled the Windows Subsystem for Linux, and this time I got Ubuntu 16.04 and along with it a much newer node.js
(good because 14.04's hella old) and a newer Unison
(not so good because now I have to find this same version for the CentOS server I use to host this site). The Unix gods, they giveth, they taketh.
Caveat Emptor: Windows 10 is not a beta, but the Windows Subsystem for Linux is. Back up everything. All the time.
Windows Subsystem for Linux and Unison update: My "old" WSL was Ubuntu 14.04, which has Unison 2.40.102 in its repository. I have Unison 2.40.102 on my CentOS server, so that worked out. Unison requires the same version on both "sides" (i.e both servers/computers) to work. My "new" WSL is Ubuntu 16.04, and that offers Unison 2.48.3.
My choices were to a) get Unison 2.48.3 for CentOS 6, or attempt to compile it on the server (or a CentOS 6 desktop, which I don't have) or b) find Unison 2.40.102 for Ubuntu 14.04.
I thought that it would be easier to compile on the server. I got the source of Unison 2.48.3, but I ran into problems pretty quickly because I needed a newer ocaml
. I was already getting in the weeds.
So I switched gears. Could I download a .deb
package from the Ubuntu repository into the WSL and install it?
I got the Unison 2.40.102 from the Ubuntu 14.04 repository. Then I used apt
to remove the Ubuntu 16.04 version of Unison.
Then I used dpkg -i
to install the .deb.
I ran unison -version
. It was working, and provided the output I wanted: Unison 2.40.102.
$ wget http://mirrors.kernel.org/ubuntu/pool/universe/u/unison/unison_2.40.102-2ubuntu1_amd64.deb
$ sudo apt remove unison
$ sudo dpkg -i unison_2.40.102-2ubuntu1_amd64.deb
$ sudo apt-mark hold unison
$ unison -version
I had already restored my .prf
file in the .unison
directory (I call it ode.prf to sync this blog), and I ran the command I use for the first sync when all the files are on the server and none on my laptop:
$ unison ode -batch
The -batch
switch lets unison
sync all of the files without asking you to OK every single one.
I love that I can get a new computer, or start a new directory and use Unison
to mirror what's on the server. More on Unison
in my next post.
I want to run Node, so I figured that I would install the package from the Ubuntu LTS in the Windows Subsystem for Linux and just use it from the Ubuntu commmand line in Windows 10.
But I soon learned that the nodejs
in the WSL is v0.10.25. That is hella old. Early 2014 old. No ES6 old.
I don't want to mess with the WSL environment too much, and I have no idea what kinds of binaries from outside the WSL will even work (if any of them will). But I wanted a newer -- a much newer -- Node.
So I installed the Windows version of Node -- the Current version -- which is v7.9.0.
That is a lot newer.
I'm not building major web applications with Node. I'm mostly using it to learn Javascript and even do some traditional scripting that I might otherwise do in Ruby or Bash.
Now I'll be doing that in the Windows command line and not the Windows Subsystem for Linux (until I can no longer hold out without a full, "modern" Linux distribution like Fedora on this laptop).
Update: Node v.0.10.25 in the Ubuntu Trusty LTS is super, super old. For comparison's sake:
Ubuntu Trusty: Node v0.10.25
Ubuntu Xenial (newer LTS): Node v4.2.6
Ubuntu Zesty: Node v.4.7.2
Debian Jessie: Node v0.10.29
Debian Stretch: Node v4.7.2
Debian Sid: Node v4.8.2
Fedora 25 and 26: Node v6.10.2
Even Debian Jessie has a slightly newer nodejs
than the Ubuntu LTS in the Windows 10 WSL. There is a way to update the Ubuntu in the WSL from 14.04 to 16.04. Might be worth a look for me.
Update: After a Windows 10 upgrade hosed the laptop, I restored Windows and reinstalled the Windows Subsystem for Linux after that. My user files were preserved, but I lost all of the files I created in the WSL.
Moral of this story: Back up your Linux files. You can back them up in your Windows user files. I would recommend making a habit of using the WSL/Ubuntu command line not in the WSL's traditional /home
directory but in your Windows user area. However, things that are complicated (and particularly which involve setting Unix-style permissions) cannot be done successfully on the Windows side. Among these "complicated" things are the use of Unison
to sync two filesystems on different computers. The Ubuntu/WSL version of Unison
works great in the WSL but throws errors aplenty when used on the Windows side. (One solution is to use the Windows version of Unison
, but I'm a whole lot of hacking away from getting ssh
working on the Windows command line in a way that Windows Unison
finds acceptable; It's not as easy as subbing PuTTY's plink
command-line tool.)
My new WSL turned out to be 16.04, not 14.04: This "solved" my Node
problem, as I got 4.2.6
instead of 0.10.25
, but I also got a newer version of Unison
(and had to download, install and "hold" the 14.04 version).
The newer Node in Windows: So I could make better use of Node in Windows, I installed the Windows version of Vim, making sure the console version was included and .bat
files were created so I could use Vim
to edit files for Node
from the Windows command line, which is somewhat of a mystery to me as I've barely used it (and many years ago at that).
This is what happens when you create a file in the Windows Subsystem for Linux and try to edit it with a Windows application
As I experiment with the Windows Subsystem for Linux (aka the Bash shell provided by Ubuntu for Windows 10), I am trying to figure exactly what I can and can't do.
To that end, I created a file with Vim in the WSL. Then I tried to open it with a text editor in Windows. I get this popup that says I can't do it:
In case you're not seeing the image above (and because Google), the Error dialog reads:
Error saving file.
Error renaming temporary file: Permission denied
The file on disk may not be truncated!
I also tried to use the Windows file manager to drop the above image, created in Windows, into the WSL portion of the disk. That file "shows" in the Windows file manager, but it doesn't appear at all in the Bash shell. I had to use Bash to copy it from the Windows side to the WSL/Linux side: That's what works, in case you were wondering.
I really need an easy drag/drop between Windows and the WSL ...
Update: This issue is addressed in a very interesting bug report with a lot of links I need to explore.
Also, in the image file I copied from Windows into Bash on Windows (as Microsoft seems to like to call it), the .jpg file was too wide open on permissions. It was 777, and I wanted 644. I made the change in Bash and am syncing with the server.
Update: While it seems fairly easy and routine to create and edit files on the Windows side of the filesystem using both Windows and WSL/Linux applications, when I tried to use the WSL-based Unison to sync files onto the Windows filesystem, I got a ton of permission errors and a failed sync. So the "dream" of maintaining a Windows system with WSL utilities probably won't happen. The two solutions for this particular problem are a) use Windows utilities on the Windows side and b) use Linux utilities on the WSL side.
The original entry begins here:
Now that I have my new HP Envy 15-as133cl laptop running Windows 10 and have added the Windows Subsystem for Linux, I'm exploring just how many of my regular Linux tasks I can do in this Ubuntu-supplied Bash shell, what I can do with similar programs compiled for Windows, and what really needs a dedicated Linux partition (or full computer).
Chief among these tasks is updating/syncing/backing up my Ode-generated blog (the one you're reading right now).
The first thing I learned about the Windows Subsystem for Linux (WSL for short) is that you can access the files you create in the WSL via the Windows file manager, but any modifications you make on the Windows side will not, I repeat WILL NOT be reflected in what you can see on the Linux side.
I really like the idea behind Less than *ambitious* websites with Ember.js, in which author sheriffderek goes through the steps required to create a simple web site with the Ember JavaScript framework.
Easing into a framework -- that's the way I want to do it.