Databases, spreadsheets, and tables - a meta discussion

A really interesting discussion started up in Telegram, and it felt far too in-depth and important to let it disappear over time there. So I’m continuing it here, and will be quoting and responding to the key messages of that discussion here. I hope everyone will continue discussing all of this here as I think it’s quite interesting and important.

This is an interesting perspective, but I don’t agree. I think simple tables are absolutely necessary/useful, at the very least for simple aesthetic formatting of non-structured content (non-structured in a “data” sense). Of course there is the column functionality of both Anytype and Notion to somewhat meet this need, and in fact I even raised the possibility of a new block formatting option for borders to help get a step closer to “tables made out of blocks”. However I still think a “simple table” block (or “markdown table”) makes sense.

Continuum or Spectrum of Functionality Concept

What I do agree with, that I think you are partly saying, is that contents needs to be able to be acted on in more structured ways at any time, as soon as it may become necessary. For that reason I think the more interesting thing to think about is a “continuum of functionality”, and an ability to in some way move in both directions along it. A capability, or a whole bunch of capabilities, to go from unstructured data, to increasing amounts of structure and functionality (this entire concept is an important meta consideration for the future of knowledge management and insight generation in general, IMO).

So for example, let’s say you start with a Simple Table. It has no functionality, no calculations, etc., it is purely aesthetic, and this is fine for a start. This is a useful thing to have. But then let’s say you realize you do want to perform some calculation or other data work on this information in this existing table. Ideally you could then very simple open the controls for the Simple Table Block that holds that data, and Convert it to a “Spreadsheet Block” or “Dynamic Table” or something. You could then do things like Sort, Calculate, and Filter. How this interacts with my request for in-line calculations I’m unsure, it could be a part of this “continuum of functionality”. Anyway, perhaps you eventually decide you now want to actually add lots of information and detail and formalized fields of data (relations) to the bits of info in your table, so again you use block options and convert it to “Table” (or “Set” perhaps) and it automatically creates Objects (pages) for every Row of that Table or Spreadsheet block, and replaces the prior block with an in-line Set view. Now all of your rows link to full pages, and each column becomes a Relation, etc.

Conversely, a non-destructive way to actually take a Table View of a Set (or a Spreadsheet block) and convert it to (ir at least view or instantiate it as) a Simple Table could be really interesting and cool too! I would imagine, due to the loss of significant functionality in the process, that you’d want it to output to a new block so as to be non-destructive. Perhaps it could even be a type of Block Reference for a Set: “Reference as Simple Table”. :thinking:

At a meta level this concept of being able to move along a “spectrum” or “continuum” of functionality while operating on the same/similar core data reminds me of some ideas and capabilities that the Fibery team is thinking about and working on. Initially those are outlined here:

And in ongoing discussion of their upcoming “blocks” functionality:

And while I think it is important to think of it in the context of “tables” as we do here, it is also important to extend this to thinking about how all content can be used and re-used in such ways (things like block references, transclusion, etc., at the least).

Yep, exactly. Although I think that A: naming these things with more familiar/intuitive concepts (as I hopefully did above under the “Continuum of functionality” heading/concept), and B: ranking them by level of functionality/sophistication, is helpful/important. So in my view that would change your #2 into a either 1 or 3 (depending on if you wanted greatest sophistication at the top or bottom), let’s say it’s #3 as it builds on simple tables. Simple tables, your #3, would be 1, and “active grid” (spreadsheet/dynamic table) would be #2. I think ordering it the way I’ve described maps more clearly to both the level of functionality available to the user, and the difficulty of implementation, both of which seem useful. Apologies if that seems pedantic, and I realize your own ordering may have some meaning to you that is lost in this reordering, so do explain if that was the case.

Well, parts of #2 already exist in the Table view of Sets. But yes, calculations don’t exist yet. That is definitely planned, however.

I don’t entirely agree, I think “simple tables” are very useful and necessary. I’d accept having to paste them from other sources (which is for example how Fibery currently works - you can paste in markdown tables, but not create or modify them within the Fibery editor), but I’d really rather be able to edit a table directly in the powerful and intuitive Anytype editor!

Not necessarily. Consider that the mere presence of additional functionality may be “cluttering”, distracting, and unwanted for people who just want “simple tables”. That said, if “Excel-like” (i.e. spreadsheet) Block in Anytype coudl be used as a Simple Table effectively, it might reduce the implementation scenarios from 3 to 2, so that is worth considering. You would still need to be able to paste in basic format-only tables, e.g. markdown tables. And it would need to be very simple to use so as not to distract or confuse anyone who just wants simple table design.

My understanding is everything is an “object”. “Document” has a particular implication that I’m not sure applies, but this may simply be a difference in terminology, our semantic intent/meaning may be the same.

Right, indeed!

Yes, I believe you are correct, generally speaking. However there is also the potentially unrelated or different concept of “blocks”, i.e. layout of objects within another object(s), on-page structural relationships (parent/child blocks), etc. And some aspects of blocks are purely aesthetic, rather than data-related, they do not hold any “value” (except if you look at the “value” for the setting of the block itself, e.g. underline text no/yes, or 0/1, color RGB value, etc.). Things like “simple tables” could be implemented in this way or could be a simple expression of a more sophisticated object-level functionality as I think you imply.

Good, let’s continue here then! :grin: I think I would just say that, while the way things work in the back end is cool and interesting to think about, without having access to the code (yet), it’s hard to be sure how it all works. Speculation may be fun but is not necessarily that useful ultimately (in my view). What matters now, IMHO, is what functionality people need, and how it works or should work, especially in terms of UI/UX, intuitiveness, ease of use, etc, etc.

Related Topics

7 Likes

This makes sense, I agree. There wasn’t any logic in my ordering (other than the order in which they came to mind, which isn’t really very helpful to anyone else :laughing:

Good news.

Thanks for organising this

1 Like

I greatly appreciate this theoretical concept of Continuum Type. However, even if Stepping-up the Continuum Type From the ground up Markdown table to a whole Set seems feasible, doing the opposite seems to me somewhat utopian. This would require the user to choose which data to keep and which to discard. Notion itself did not consider the ability to downgrade a Table Database to a simple Table, probably for this very reason.

I’d say that with the exception of the time impact, what once evolve can hardly devolve, sort of.

I guess that without a concrete mockup from @yopp initial vision, I’m currently stuck in a black hole of misty speculations. Unfortunately, despite his declaration, he did not show much sign of life on the forum to date.

1 Like

That’s only true based on assumptions you make. There are actually numerous ways you could handle this differently/better, some of which Notion actually models (I’ll get to that in a moment), others of which are, I think, interesting to consider in themselves. Here are a few ideas on how it could work (some of which I already covered above). Starting with the premise that you have a set of “table” data that is at a maximal level of complexity, i.e. a database with pages “inside of” or connected to it, and some Properties with data in them too.

Block History Approach

One interesting idea I’ve written about before is the possibility of having block-level history. If you did have that, you could simply use the history to revert to a full database representation if needed. Obviously there are potential complexities in how you handle “child” pages given the complexity of a DB, but I’m sure we could come up with some interesting and probably workable solutions.

Blocks as Non-Destructive Data Containers (Notion-like)

Another approach, which I believe is kinda how Notion works, at least with some block types, is to actually keep data in the block description itself, internally, even when you transform that block to a type that does not support that data. In other words you can transform blocks in Notion from, say, a Code Block, where you’ve made a language selection (what language will be used for syntax highlighting), and turn it into e.g. a Header, where the “language selection” data is meaningless, but if you then turn that block back into a code block, the language selection will be maintained. My understanding is essentially that any field with data in it is maintained inside the block, whatever you turn it into, and each block type just displays fields or formatting, etc. that is appropriate for its block type. So if you change the background color of a block, any block type you turn it into can still be highlighted, and even if you turn it into one that can’t, if you change it back to one that can, the highlight should just show up again because the value of it was maintained in the data structure of the block.

Anyway, long story short, you could use a similar approach to maintain any data, such as Field Properties and Values, etc. of a database, if you turned it into a Simple Table. Obviously this makes for much “heavier” data in the block, but it does not seem so hard to handle. Probably a simple reference to the table data or links or properties or whatever, that is simply checked and not traversed if it is anything but a “Database” block type.

“Views” and Transclusion

A final option that comes to my mind, that I already mentioned above, is one of simply using a transclusion (reproduce the contents of a block in another place), or even just the concept of “views” like Notion uses, and simply extend it by adding another view “Simple Table”. This is probably the most limited approach but also likely the easiest to implement. You simply create a view of data, that can be used either on a Database/Table/Set/whatever, or on a Tarnsclusion of a block/data, which has an extremely minimal “simple table-like” design and make it function as much like a simple table as you can. You should be easily able to support things like dragging rows and columns, and various other “simple table-type” options.

Challenges

The obvious problem with at least some of these options is how to handle entry of data in one type of representation that is not compatible with another type. You’d need some sensible rules for this, and potentially to give the user some prompts to make choices how they want to handle it. This connects with your “choose what data to keep and discard” concern, although I think that is a simplistic and overly dramatic example which in practice is not necessary with most of the 3 options I’ve outlined above. What might however be necessary is conversion of data, which is not so difficult I think. Worst case yes the data might be “discarded” in certain cases, but block-level history and blocks as non-destructive data containers could solve that, albeit perhaps imperfectly. The gain in utility would, I think, potentially be worth it.

This is actually not correct. You can, in fact, convert a Database back to a Simple Table… as long as you don’t add any data to it that is not compatible with Simple Table. :grinning_face_with_smiling_eyes: This creates some notable limitations, to be sure, but they wouldn’t have provided the option if they didn’t see some utility for it, eh? I talked a bit about it and linked to some demo videos in an update in my digital garden:

Good point, I would love to see them jump back in to this discussion!

3 Likes

I would reply that proof of presence is not a presence of proof :wink:


I see that you don’t seem to be lacking in ideas about data retention, but I didn’t see any Cornelian choices about it in the first place. On the other hand I must say that your comment in your “Challenges” section about possible efficiency trade-offs worries me.

Because if I use a block conversion option, I have to be able to trust it blindly.


Your expertise is certainly a guarantee of technical insight. In comparison I am just a power-user, but I like to think that on this particular point, you seem to be downplaying the impact of the issues raised by these backward data conversions.

2 Likes

Perhaps, perhaps. My goal here is really just to discuss the possibilities and see if any highly valuable-seeming capabilities identify themselves. I’m not expert enough to really assess technical feasibility fairly, so I approach it more from a “what would be really useful to have, regardless of possibility” perspective, and then hopefully work backward from there to what is practical now, ideally thinking along with the devs helping to clarify what is reasonable to implement.

2 Likes