Cygwin Bash as a REPL inside The Console

In previous blogposts I discussed scripting custom REPLs for The Console. This time the custom will be just a good ol’ Unix shell called bash.

Agenda

1. bind bash so it will respond to my commands and print results to output window
2. try out auto-completion for command names

Please note, I use Windows, so I’ll bind bash which comes with Cygwin.

Just bind that bash, already

var InputStreamReader = Java.type("java.io.InputStreamReader")
var BufferedReader = Java.type("java.io.BufferedReader")
var ProcessBuilder = Java.type("java.lang.ProcessBuilder")
var Thread = Java.type("java.lang.Thread")

var CYGWIN_DIR = "N:/.babun/cygwin/"

var pb, process, inputThread, keepAlive

exports.commandLineHandler = {
	init: function() {
		pb = new ProcessBuilder(CYGWIN_DIR + "bin/bash.exe")
		pb.redirectErrorStream(true)

		var env = pb.environment()
		env.put('Path', env.get('Path') + ";" + CYGWIN_DIR + "bin")
		
		process = pb.start()
		
		keepAlive = true
		inputThread = new Thread(listenToInput.bind(null, this.context))
		inputThread.start()
	},

	handleExecution: function(input, utils, context) {
		var os = process.getOutputStream()

		// display in console what a user typed in
		context.output.addInputEntry(input)

		// send input text to bash
		os.write(input.getBytes())
		os.write(10) // ENTER
		os.flush()
	},
	
	dispose: function() {
		keepAlive = false
		inputThread = null
	}
}

Boom.

The main trick here is to use java.lang.ProcessBuilder which will start bash.exe  and provide us with 3 streams: input, output, error. However, I call redirectErrorStream to merge the error stream into output stream for simplicity.

What we lack here is definition of listenToInput() function. By the way, note the nomenclature here. While process have input stream that receives input from user, it’s named as output stream because from ProcessBuilder  perspective the input data is pushed rather than pulled. The same goes with bash output (that gathers results of invoked commands) which from ProcessBuilder  perspective is input, because we’re going to read that, not opposite. It’s all reversed.

So, let’s listen to the ProcessBuilder  input (output + errors from bash):

function listenToInput(consoleContext) {
	var input = new BufferedReader(new InputStreamReader(process.getInputStream(), "UTF-8"))
	var output = consoleContext.getOutput()
	
	while (keepAlive) {
		if (!process.isAlive()) {
			// this can be achieved by invoking `exit` command
			output.addErrorEntry("bash process is dead.")
			
			// bring back standard handler
			consoleContext.commandLineService.resetHandler()
			return
		}
		
		while (input.ready()) {
			output.addTextEntry(input.readLine())
		}
		
		// don't consume too much CPU
		Thread.sleep(50)
	}
}

Nothing special here. Whole thing runs in a thread. What bash outputs is being read and print on The Console -  line by line.

Why async?

There are at least two ways to do all of this:

1. synchronously
2. asynchronously

Above, you’ve seen the second. Why? Simplicity.

As you may notice, input and output streams are processed in two different threads. handleExecution() is called directly from some thread of The Console (probably JavaFX thread), while listenToInput()  works on another thread that is started by hand in init() .

The best and bullet-proof way would be to implement synchronous approach on top of queues. Why queues? Because of command completion.

Command name completion

pw[TAB]

this little scenario above is what we call auto-completion in bash. The result will be just this in command line:

pwd

And that’s my aim. Being in bash I want to only list command names that start with letters I already typed. Let’s extend our commandLineHandler  with completion:

handleCompletion: function(input) {
	var os = process.getOutputStream()
	var toComplete = 'compgen -c ' + input
	os.write(toComplete.getBytes())
	os.write(ENTER)
	os.flush()
},

And our scenario will work:

The handleCompletion()  implemented above only prints output from calling compgen -c pw .  It would be nice to change input text field, as auto-completion would do. However, output is collected on another thread so I don’t have output for that exactly input. That’s why I would need to synchronize management of input and output.

Queue: synchronize output with input

The main idea is queue every kind of input. In this situation it’s a input pushed on ENTER key and completion input pushed on TAB key:

handleCompletion: function(input) {
	inputQueue.push({
		type: INPUT_TYPE_COMPLETION,
		text: input
	})
},

handleExecution: function(input, utils, context) {
	// display in console what a user typed in
	context.output.addInputEntry(input)
		// queue request for execution
	inputQueue.push({
		type: INPUT_TYPE_CMD,
		text: input
	})
},

The remaining job is to process all those inputs in a Thread. I won’t go into details but you can find full example code here -> The Console Wiki - bash REPL.

Summary

The plan of having a really custom REPL succeeded. This way I achieved an access to really big library of commands. For example, instead implementing custom md5 file hashing command, I could simply use cygwin built-in md5sum .

Auto-completion provided above is really simple. However, it’s not limited to this functionality. Bash supports programmable auto-completion of arguments for any command, however this is a a little bigger task to implement calling complete  command for argument completion.

References

1. An introduction to bash completion
2. NamekDev: Scriptable REPL in The Console
3. NamekDev: Argument completion for commands in The Console
4. The Console Wiki - bash REPL