When writing answers for Stack Overflow it is a good idea to test your code before publishing it. In some cases to replicate the questioner's problem I have to install extra packages.
I find it very useful for cases like this to use Google Colab to quickly install the packages I need, write code, test it, and finally write an answer, all without having to install anything locally on my computer. I then manually copy and paste the Colab code into SO, but that can introduce errors or inconsistencies.
Ideally, you would be able to develop a full, tested response in Colab and then automatically somehow "export" what was written there for inclusion in an SO response.
As far as I know in Jupyter Notebook there is a command to export to markdown, is there an automatic way to achieve the same in Google Colab, considering that there is no access to the file .ipynb
?
A Jupyter Notebook can be converted to markdown, using the command:
The result is a file extension
.md
quite similar to what was indicated in the question. In that file the text cells have been converted to markdown (compatible with the one used by Stack Overflow), the code cells have been dumped as is but delimited by the triple tick (```) that makes them look correctly in Stack Overflow, and the cell outputs (the output) are displayed as is but indented four spaces, which will cause Stack Overflow to display them as code as well.The problem is that this command must be executed from the command line, which requires Jupyter to be installed (and the notebook to be exported previously downloaded from Colab).
The problem with Colab
Colab has a somewhat peculiar architecture. A Jupyter notebook is on the one hand a file (typically with the extension
.ipynb
), but on the other hand it is more than just a file, since it is "alive" (code can be added to it, it can be executed and the result can be seen). To enable the latter, a machine running a Python interpreter (what Jupyter calls the execution kernel ) is needed, which receives the contents of the cell to be executed and returns the result.Google Colab separates, on the one hand, the storage of the .ipynb file (which is saved in the user's Google Drive unit) and, on the other hand, the execution of the .ipynb (which occurs in a virtual machine that Google starts and that contains python preinstalled, a Jupyter runtime kernel, and lots of typical scientific packages). That virtual machine is "ephemeral" (if you don't use it for a while it will be destroyed), but while it exists you can install things on it and it has its own file system. And when you reopen the notebook, if the machine had been shut down, a new one will be created again.
From a Jupyter notebook "system" commands preceded by a can be executed
!
, in the case of Colab and these commands are executed in the virtual machine described above.So we could think that the command
jupyter
shown above could also be executed from the notebook itself, putting in a cell:But the problem is that the file
.ipynb
is not on the disk of the virtual machine (or at least it is not in any folder accessible from the notebook itself), but it is stored in Google Drive, which complicates things.The solution
It is necessary to insert a cell that, through Python code, allows the connection between the notebook and the Google Drive, to transfer a copy of the .ipynb of the Drive to the virtual machine from which the command can already be executed
!jupyter nbconvert
.The following code (whose operation I will explain next) would do everything necessary:
The first line with
# IGNORE
, although it is a comment, is there to prevent this cell from also being part of the resulting markdown, as we will see later.The line with
# @title
is to make this cell "collapsible" in Colab, so we don't see its code (by double-clicking on the cell title), which makes the workbook cleaner.The mapping
notebook_name = "Pruebas.ipynb"
should be changed to the name of the Colab notebook you want to convert (it can be the same as the one in this cell, or something else).The part that says "Connecting to Drive" will cause a URL to appear for the user to connect to, which will take them to OAuth2 authorization which will grant this notebook permissions to access Drive. From that page you will get a code that you must then paste into an input to continue. Once that is done, and as long as this notebook is not shut down, you will not be prompted for this authorization again. This cell can be executed several times if modifications are made to the notebook and the new markdown version is to be obtained. Only the first time authentication is required.
The part that says "Download the notebook" deals with searching Drive for the document with the name that we have assigned in
notebook_name
, and downloading it to the virtual machine in which the kernel of this notebook is running.Finally, the command is executed
jupyter nbconvert
on that virtual machine, with the appropriate options to generate the desired markdown, and to skip the cells that start with# IGNORE
. The result of that conversion is sent to standard output ( option--stdout
) from which it is captured by the notebook to store in the variabletxt
.This variable is a list of strings, each string being each of the lines that the command has produced. The first of those lines is a message from nbconvert, which isn't really part of the output, so we skip it, and the rest we concatenate with newlines. Finally we print
txt
The result of all of the above is that the markdown version of the notebook is displayed as the output of this cell. That text is ready to be copied (Ctrl+A to select, Ctrl+C to copy) and pasted into SOes (Ctrl+V).
warnings
Some things don't work directly:
![asi](fichero.png)
that won't work correctly on OSes. Graphics must be copied and pasted separately into SOes.print(df)
(instead of justdf
) so that the output is ASCII instead of an HTML table.show
The following image shows an example notebook as seen in Colab:
The last cell of that notebook contains the code explained above. Executing that cell produces the following output, which is the contents of the notebook in markdown, ready for copy and paste:
If I copy and paste that text into SOes, I'll have to manually edit the line that contained the image, which read:
and put in its place the image that I can copy and paste directly from the cell that produced it.
The result would be the following:
Example
This cell contains markdown , and is a proof of concept .
Now an unnumbered list
End of cell markdown.
An example showing a graph
An example showing a table (using
print()
to avoid HTML output)