Tag: web

Stephen Wolfram on ChatGPT

Published April 4, 2023 by lievenlb

A month ago, Stephen Wolfram put out a little booklet (140 pages) What Is ChatGPT Doing … and Why Does It Work?.

It gives a gentle introduction to large language models and the architecture and training of neural networks.

The entire book is freely available:

The advantage of these online texts is that you can click on any of the images, copy their content into a Mathematica notebook, and play with the code.

This really gives a good idea of how an extremely simplified version of ChatGPT (based on GPT-2) works.

Downloading the model (within Mathematica) uses about 500Mb, but afterwards you can complete any prompt quickly, and see how the results change if you turn up the ‘temperature’.

You should’t expect too much from this model. Here’s what it came up with from the prompt “The major results obtained by non-commutative geometry include …” after 20 steps, at temperature 0.8:

NestList[StringJoin[#, model[#, {"RandomSample", "Temperature" -> 0.8}]] &, "The major results obtained by non-commutative geometry include ", 20]

The major results obtained by non-commutative geometry include vernacular accuracy of math and arithmetic, a stable balance between simplicity and complexity and a relatively low level of violence.

Lol.

In the more philosophical sections of the book, Wolfram speculates about the secret rules of language that ChatGPT must have found if we want to explain its apparent succes. One of these rules, he argues, must be the ‘logic’ of languages:

But is there a general way to tell if a sentence is meaningful? There’s no traditional overall theory for that. But it’s something that one can think of ChatGPT as having implicitly “developed a theory for” after being trained with billions of (presumably meaningful) sentences from the web, etc.

What might this theory be like? Well, there’s one tiny corner that’s basically been known for two millennia, and that’s logic. And certainly in the syllogistic form in which Aristotle discovered it, logic is basically a way of saying that sentences that follow certain patterns are reasonable, while others are not.

Something else ChatGPT may have discovered are language’s ‘semantic laws of motion’, being able to complete sentences by following ‘geodesics’:

And, yes, this seems like a mess—and doesn’t do anything to particularly encourage the idea that one can expect to identify “mathematical-physics-like” “semantic laws of motion” by empirically studying “what ChatGPT is doing inside”. But perhaps we’re just looking at the “wrong variables” (or wrong coordinate system) and if only we looked at the right one, we’d immediately see that ChatGPT is doing something “mathematical-physics-simple” like following geodesics. But as of now, we’re not ready to “empirically decode” from its “internal behavior” what ChatGPT has “discovered” about how human language is “put together”.

So, the ‘hidden secret’ of successful large language models may very well be a combination of logic and geometry. Does this sound familiar?

If you prefer watching YouTube over reading a book, or if you want to see the examples in action, here’s a video by Stephen Wolfram. The stream starts about 10 minutes into the clip, and the whole lecture is pretty long, well over 3 hours (about as long as it takes to read What Is ChatGPT Doing … and Why Does It Work?).

Loading a second brain

Published March 6, 2023 by lievenlb

Before ChatGPT, the hype among productivity boosters was a PKMs or Personal knowledge management system.

It gained popularity through Tiago Forte’s book ‘Building a second brain’, and (for academics perhaps a more useful read) ‘How to take smart notes’ by Sönke Ahrens.

These books promote new techniques for note-taking (and for storing these notes) such as the PARA-method, the CODE-system, and Zettelkasten.

Unmistakable Creative has some posts on the principles behing the ‘second brain’ approach.

Your brain isn’t like a hard drive or a dropbox, where information is stored in folders and subfolders. None of our thoughts or ideas exist in isolation. Information is organized in a series of non-linear associative networks in the brain.

Networked thinking is not just a more efficient way to organize information. It frees your brain to do what it does best: Imagine, invent, innovate, and create. The less you have to remember where information is, the more you can use it to summarize that information and turn knowledge into action.

and

A network has no “correct” orientation and thus no bottom and no top. Each individual, or “node,” in a network functions autonomously, negotiating its own relationships and coalescing into groups. Examples of networks include a flock of birds, the World Wide Web, and the social ties in a neighborhood. Networks are inherently “bottom-up” in that the structure emerges organically from small interactions without direction from a central authority.

-Tiago Forte, Tagging for Personal Knowledge Management

There are several apps you can use to start building your second brain, the more popular seem to be Roam Research, LogSeq, and Obsidian.

These systems allow you to store, link and manipulate a large collection of notes, query them as a database, modify them in various ways via plugins or scripts, and navigate the network created via graph-views.

Exactly the kind of things we need to modify the simple system from the shape of languages-post into a proper topos of the unconscious.

I’ve been playing around with Obsidian which I like because it has good LaTeX plugins, powerful database tools via the Dataview plugin, and one can execute codeblocks within notes in almost any programming language (python, haskell, lean, Mathematica, ruby, javascript, R, …).

Most of all it has a vibrant community of users, an excellent forum, and a well-documented Obsidian hub.

There’s just one problem, I’m a terrible note-taker, so how can I begin to load my ‘second brain’?

Obsidian has several plugins to import data, such as your Kindle highlights, your Twitter feed, your Readwise-data, and many others, but having been too lazy in the past, I cannot use any of them.

In fact, the only useful collection of notes I have are my blog-posts. So, I’ve uploaded NeverEndingBooks into Obsidian, one note per post (admittedly, not very Zettelkasten-like), half a million words in total.

Fortunately, I did tag most of these posts at the time. Together with other meta-data this results in the Graph view below (under ‘Files’ toggled tags, under ‘Groups’ three tag-colours, and under ‘Display’ toggled arrows). One can add colour-groups based on tags or other information (here, red dots are posts tagged ‘Grothendieck’, the blue ones are tagged ‘Conway’, the purple ones tagged ‘Connes’, just for the sake of illustration). In Obsidian you can zoom into this graph, place a pointer on a node to highlight the connecting dots, and much more.

Because I tend to forget such things, and as it may be useful to other people running a WordPress-blog making heavy use of MathJax, here’s the procedure I followed:

1. Follow the instructions from Convert wordpress articles to markdown.

Export your WordPress posts to XML
Install Node.js
Run the wordpress-export-to-markdown script

In the wizard I’ve opted to go only for yearly folders, to prefix posts with the date, and to save all images.

2. This gives you a directory with one folder per year containing markdown versions of your posts, and in each year-folder a subfolder ‘img’ containing all images.

Turn this directory into an Obsidian-vault by opening Obsidian, click on the ‘open another vault’ icon (third from bottom-left), select ‘Open folder as vault’ and navigate to your directory.

3. You will notice that most of your LaTeX cannot be parsed because during the markdown-process backslashes are treaded as special character, resulting in two backslashes for every LaTeX-command…

A remark before trying to solve this: another option might be to use the wordpress-to-hugo-exporter, resulting in clean LaTeX, but lacking the possibility to opt for yearly-folders (it dumps all posts into one folder), and it makes a mess of the image-files.

4. So, we will need to do a lot of search&replaces in all files, and need a convenient tool for this.

First option was the Sublime Text app, which is free and does the search&replaces quickly. The problem is that you have to save each of the files, one at a time! This may take hours.

I’ve done it using the Search and Replace app (3$) which allows you to make several searches/replaces at the same time (I messed up LaTeX code in previous exports, so needed to do many more changes). It warns you that it is dangerous to replace strings in all files (which is the reason why Sublime Text makes it difficult), you can ignore it, but only after you put the ‘img’ folders away in a safe place. Otherwise it will also try to make the changes to these files, recognise that they are not text-files, and drop them altogether…

That’s it.

I now have a backup network-version of this blog.

As we mentioned in the previous post a first attempt to construct the ‘topos of the unconscious’ might be to start with a collection of notes (the ‘conscious’) and work on the semantics of text-snippets to unravel (a part of) the unconscious underpinning of these notes. We also mentioned that the poset-structure in that post should be replaced by a more involved network structure.

What interests me most is whether such an approach might be doable ‘in practice’, and Obsidian looks like the perfect tool to try this out.

What we need is a sufficiently large set of notes, of independent interest, to inject into Obsidian. The more meta it is, the better…

(tbc)

Previously in this series:

Next:

The enriched vault

mathblogging and poll-results

Published January 21, 2011 by lievenlb

Mathblogging.org is a recent initiative and may well become the default starting place to check on the status of the mathematical blogosphere.

Handy, if you want to (re)populate your RSS-aggregator with interesting mathematical blogs, is their graphical presentation of (nearly) all math-blogs ordered by type : group blogs, individual researchers, teachers and educators, journalistic writers, communities, institutions and microblogging (twitter). Links to the last 7 posts are given so you can easily determine whether that particular blog is of interest to you.

The three people behind the project, Felix Breuer, Frederik von Heymann and Peter Krautzberger, welcome you to send them links to (micro)blogs they’ve missed. Surely, there must be a lot more mathematicians with a twitter-account than the few ones listed so far…

Even more convenient is their list of latest posts from their collection, ordered by date. I’ve put that page in my Bookmarks Bar the moment I discovered it! It would be nice, if they could provide an RSS-feed of this list, so that people could place it in their sidebar, replacing old-fashioned and useless blogrolls. The site does provide two feeds, but they are completely useless as they click through to empty pages…

While we’re on the topic of math-blogging, the results of the ‘What should we write about next?’-poll that ran the previous two days on the entry page. Of all people visiting that page, 2.6% left suggestions.

The vast majority (67%) wants more posts on noncommutative geometry. Most of you are craving for introductions (and motivation) accessible to undergraduates (as ‘it’s hard to find quality, updated information on this’). In particular, you want posts giving applications in mathematics (especially number theory), or explaining relationships between different approaches. One person knew exactly how I should go about to achieve the hoped-for accessibility : “As a rule, I’d take what you think would be just right for undergrads, and then trim it down a little more.”

Others want rather specialized posts, such as on ‘connection and parallel transport in noncommutative geometry’ or on ‘trees (per J-L. Loday, M. Aguiar, Connes/Kreimer renormalization (aka Butcher group)), or something completely other tree-related’.

Fortunately, some of you told me it was fine to write about ‘combinatorial games and cool nim stuff, finite simple groups, mathematical history, number theory, arithmetic geometry’, pushed me to go for ‘anything monstrous and moonshiney’ (as if I would know the secrets of the ‘connection between the Mathieu group M24 and the elliptic genus of K3’…) or wrote that ‘various algebraic geometry related posts are always welcome: posts like Mumford’s treasure map‘.

5 Comments

lulu neverendingbooks

Published July 18, 2005 by lievenlb

Half a year ago, it all started with NeverEndingBooks in which I set out a rather modest goal:

Why NeverEndingBooks? We all complain about exaggerated prices of mathematical books from
certain publishers, poor quality of editing and refereeing offered, as well as far too stringent book-contracts. Rather than lamenting about this, NeverEndingBooks gives itself one year to learn (and report) about the many aspects of the book-production cycle and to explore whether an alternative exists. If at the end of this year we will have produced at least one book this experiment will be considered a success. If,
however, we find out that it is an impossible task, we will explain where it all went wrong and why it is better to stick to an established PublishingHouse and accept its terms.

I assume we did manage to do it after all as you may check by visiting our storefront :
www.lulu.com/neverendingbooks. However, it all turned out to be quite different from what I had in mind half a year ago. So, perhaps it’s time to recap.

Originally, I’d planned to partner-up with the publisher-on-demand LightningSource but in the process they did require a VAT-number. In Belgium, the safest way to get one is to set up a non-profit organization (a VZW as we call
it). But then you have to write down your legal statutes, get them published in the Moniteur Belge (at a hefty price) but what really put me off was that you have to set up a “board of directors” consisting of at least three
people. I don’t mind following a folly but if I have to involve others I usually pass, so I abandoned the whole idea.

Still, I couldn’t help talking about the VAT-problem and at a certain time there was an idea to revive a sleeping VZW (=non-profit organization) of the Belgian Mathematical Society, the MaRC (MAthematical Research Centre), the statutes of which allowed to become a publishing house. But, this wouldn’t involve just two other people but the whole BMS so I decided
to forget all about it and have a short vacation in France together with a few (former)PhD-students.

Given plenty of sun, cheese and whine (not necessarily in that order) sooner or later we had to talk about _the_ problem. For Raf it was the first time he heard about it but when we realized I thought one could easily publish books well under 25 dollars he was immediately interested and insisted we should set up a board of directors and continue with the plan.

The different roles to play in the board were more or less self-evident : I had to be the treasurer (given the fact that I was the only with a secure, though small, income), Geert had to become chairman (being the only one possessing suits), Raf would be secretary (being the only one who could write better Flemish than English) and Jan or Stijn would do PR (as they are the only ones having enough social skills).

So, we went back willing to go through the whole process (at least 3 months) of obtaining a VAT-number.

But then Raf got so interested in the whole idea that he explored other possibilities (I think he was more motivated by the fact that his sister wanted to publish her thesis rather than anything else) and came up with lulu.com.

No legal hassle, no VAT-numbers, nothing required (or so it seemed). Still, before risking his sister’s thesis he wanted to check the service out and as it is a lot easier to take a book lying around rather than write one yourself he took my version 2 and published it at Lulu’s (since then this version is nicknamed Rothko@n).

Although I gave him the permission to do so, it didn’t feel right that people should pay even a small amount for a nicely bound unedited version 2. So, the last month and a half I’ve been editing and partially rewriting version 2 and the two volumes are now available!

Major changes are to the 4 middle chapters. There is now chapter 3 “Etale Technology” which contains all of the etale tricks scattered earlier in two chapters, chapter 4 ‚”Quiver Representations” collects all the
quiver material (again, scattered throughout the previous version). Chapter 5 ‚”Semisimple Representations” now includes recent material such as Raf’s characterization of the smooth locus of Cayley-smooth orders and our (together with Geert) classification of the central singularities, and chapter 6 ‚”Nilpotent Representations” now includes the material on Brauer-Severi varities which was in version 1 but somehow didn’t make it to version 2 before.