Merged Alex's IRI support, with tiny changes.

[elpher.git] / ISSUES.org

#+TITLE: Issues and Dev Notes
#+TODO: OPEN(o!) | CLOSED(c!) INVALID(i@)
#+STARTUP: logdrawer
   
* Open Bugs

** OPEN Sanitize certificate names
   :LOGBOOK:
   - State "OPEN"       from              [2020-06-22 Mon 10:32]
   :END:
   
Currently things will break in undefined ways if a name is specified
that contains path separators and probably other characters that I
haven't thought of.  This is dangerously unacceptable and needs to be
fixed right away.

** OPEN Set timer after creating network process

While the current order is necessary for synchronous socks
connections, it is unecessary for regular connections which have the
no-wait flag set.  Furthermore, for these connections, having the
timer fire up early means that it interferes with requests for
user interaction that may appear during the initial connection setup.
E.g., asking for approval of uknown TLS certificates.

* Closed Bugs

** CLOSED Relative Gemini links processed improperly
:LOGBOOK:
- State "CLOSED"     from "OPEN"       [2021-08-04 Wed 15:54]
- State "OPEN"       from              [2021-08-04 Wed 13:53]
:END:

Skyjake's gemlog at gemini://skyjake.fi/gemlog/ demonstrate's the
issue.  The link back to the root selector in the footer of that page
is a relative link to the parent directory, i.e. "..".  For some
reason elpher combines this with the current URL and produces
"gemini://skyjake.fi" as the destination of the link.  Such URLs
(i.e. without a filename) are allowed as input, but are assumed
to not appear internally.

To see why the internal distinction is important, consider a page
where the current URL is gemini://example.com/a_page.  The current
directory in this case is "/", meaning a relative link to
"another_page" results in a destination link of
"gemini://example.com/another_page.  On the other hand, if the current
URL is gemini://example.com/a_page/, the same relative link is
interpreted as refering to gemini://example.com/a_page/another_page.

The fix will be to ensure gemini://skyjake.fi/gemlog/.. collapses to
gemini://skyjake.fi/ rather than gemini://skyjake.fi.


  
** CLOSED Org mode faces are not present in recent emacs versions
Even 26.1 doesn't seem to have these.  This means that, for many
users, elpher doesn't show any difference between any of the
item types.  Not a major problem at all, but the faces we inherit
from should definitely be ones which have been present for much
longer.  Perhaps the font lock mode faces are the way to go after
all.

Update: changed all default faces to inherit from font-lock and basic faces.

** CLOSED URL-centric addressing breaks bookmark file compatibility
   
Need a way to allow people to rescue their old bookmark files
following this update.

** CLOSED History loops <2019-11-08 Fri>

Occasionally elpher gets stuck in a "history loop" where a
node is its own grandparent.  Obviously this sucks, as history
is elpher's main mechanism for making gopherspace exploration
painless.

I suspect the problem is in either ~elpher-visit-node~ or
~elpher-visit-parent~.

Follow-up: this has been fixed by the new stack-based history system
in 2.5.


** CLOSED Redirects do not rewrite current address

This is a bug, as gemini://blah.com/hi may get redirected
to gemini://blah.com/hi/, at which point link lines
of the form "=> there" should be interpreted as pointing
at gemini://blah.com/hi/there, while currently they are
interpreted as pointing at gemini://blah.com/there.

** CLOSED History inconsistency when restarting elpher <2020-05-26 Tue>

To reproduce:
1. open elpher and follow a few links until you're a handful of links below
   the start page.
2. kill the elpher buffer with C-x k
3. Open elpher again, which will show the start page.
4. Press 'u' to go up.  Elpher wiill respond stating that there is no previous page.
5. Press 'u' again. Elpher will then jump to the page that was open when
   the buffer was originally killed.

Expected behaviour: elpher should be once again at the bottom of the history
stack and should not remember the previous history.

Observed behaviour: elpher _does_ remember the previous history.

*** update <2020-05-27 Wed>
Turns out this was just because the `elpher` function was merely setting
the `elpher-current-page` variable to nil, then using `elpher-visit-page`
to visit the start page, resulting in the nil being pushed onto the existing
history stack.  Because `elpher-visit-previous-page` always trys to pop from
this stack and tests whether the result is nil (which it is when the stack is empty),
the first "u" would result in the "no previous page" message but would still
pop the stack, meaning that subsequent "u" commands would succeed.

The fix is just to zero out the history list in the `elpher` function just as
`elpher-current-page` is cleared.

* Open Enhancements

** OPEN Allow multiple elpher buffers [33%]

   Shouldn't be too hard, just need elpher-current-node to be
buffer-local and allow various buffer-switching procedures to
do something sensible.

Here are the things that need to be implemented before
this can happen:
- [X] shift history out of node tree and into separate stack
- [ ] make history stack variables buffer-local
- [ ] have elpher-with-clean-buffer select appropriate buffer 

** OPEN Replace support for user-specified starting pages
This used to be available, but was removed during a refactor.
   
** OPEN Make installing existing certificates easier
   :LOGBOOK:
   - State "OPEN"       from "CLOSED"     [2020-06-22 Mon 10:34]
   :END:

It's naive to think that people don't have client certificates created
outside of elpher. Thus we need some easy way to "install" these
certificates, either by copying them or by referencing them in some
way.

* Closed Enhancements
  
** CLOSED Turn on lexical scoping

   A branch exists for this, but there are some compilation kinks
to iron out.

  
** CLOSED Implement support for telnet entries

Similar to http entries, telnet entries will be handled by code
external to elpher. However it seems I made http entry handling a
special case, and I don't want another!  So the only option is to
bring both http and telnet entries back into the fold by representing
them both as standard nodes and having the grunt work done by getter
functions.

** CLOSED Allow users to access selected and current node details.
   
** CLOSED Implement bookmark system

  Currently the bookmark page replaces the current page, and it
  does so silently (i.e. it doesn't become part of the link hierarchy).
  I think this is a mistake, as it results in confusing behaviour when
  traversing the link hierarchy after visiting one of the bookmarked links.

  Instead, I think I should
  1. Make the bookmark page part of the hierarchy, and
  2. Reinstate the visited node hash table to avoid excess link hierarchy pollution.

  In order to accomplish 1. it will be necessary to make the bookmark page renderer
  a proper getter function, and one that never caches the contents of the buffer.

  Actually, I might have to think about that a bit more.  I don't know
  how to answer the question of what the best thing to do with node
  parent links when using a cached node in place of a new node.  (Maybe
  I always update node.parent unless parent is already an ancestor of
  node?)

  
** CLOSED Support character encoding diversity

** CLOSED Make URLs the basic address type.
Currently I waste a lot of effort converting between
URL and non-URL representations.  This is unnecessary, and
actually makes lots of things uglier.

For example, the bookmarks file contains addresses in Elpher's
internal representation, whereas I expect users would prefer
it contain URLs.

So the idea would be for (elpher-node-address node) to be
a either a string or a symbol, with symbols used for "special"
pages (bookmarks, start page, etc).  The getter functions
`elpher-address-selector' etc will still do what they currently
do, but will process the URL to do it.

This also means that non-gopher URLs will be explicitly represented
as such: no more abusing the "h" type for these.

** INVALID Remove "redraw" command
This is only necessary for returning from displaying the raw
server response.  If I can provide a better way of doing that
then we can get rid of redraw entirely.

Actually, this command can be useful to correct rendering issues that
occasionally pop up in termal windows.  Lets leave it for now.

** CLOSED Implement Finger support
   
** CLOSED Improve download performance
   This is actually easy to fix - the major problem at the moment is
   the braindead way the incrementally-retrieved data is recorded:
   (setq result-string (concat result-string next-bit)).
   This is O(N^2).  Yuck!
   
   Okay, replacing this really does improve things.  Large gemini
   downloads now seem occur at rates I'd expect.
   
** CLOSED Download/rendering progress feedback
   Particularly for large files or complicated pages, elpher can
   take a few seconds or more to generate a response.  Thhis is
   frustrating for users, who are left staring at a blinking
   cursor.

   A small amount of feedback could help with this.

** CLOSED Implement Gemini support [100%]
   :LOGBOOK:
   - State "CLOSED"     from "OPEN"       [2020-06-20 Sat 22:32]
   :END:
   
Here is the checklist of features required before release:
- [X] basic genimi transactions
- [X] gemini transactions requiring client certificates
- [X] gemini input handling
- [X] gemini map files (text/gemini)
- [X] Support for plain text responses (text/*)
- [X] Support for image responses (text/image)
- [X] Support for mime-specified character encodeing
- [X] Saving responses to disk
- [X] Viewing raw responses
  
The last few will be made infinitely easier if we factor the
gopher "getter" code differently.


** INVALID Allow for grouping of bookmarks
:LOGBOOK:
- State "INVALID"    from              [2021-07-23 Fri 10:10] \\
  Since switching to Emacs native bookmarks, this is no longer our concern.
:END:
To support this I'd like to add a bookmark page specific
set of keybindings.  Currently all bindings available on
the bookmark page are available everywhere else.  But
expanding and collapsing bookmark groups sounds like it
might need more specific bindings.

*** Priority bump <2020-05-31 Sun>

As bookmark lists grow, some sort of grouping is becoming more and more
important.  Furthermore, with this in place it would become feasible
(and I really suspect almost trivial) to implement an update-checking
system for chosen groups of bookmarks.

For instance, we could prefetch content for each of the addresses within
a chosen group, indicating which had been changed since the last fetch.
(We could just store hashes of earlier content to detect changes.)

The difficult thing to decide is how the UI for the new bookmark page
will work.  It already has its own renderer, and we could easily stop
using the gopher directory line renderer in favour of something more
amenable to displaying the group information.  Thus we're very free to
do whatever we like once we also have a special key map in place as well.

I guess I need to look into what native widgets Emacs has for displaying
collapsable hierarchies.


** CLOSED Add history browsing
:LOGBOOK:
- State "CLOSED"     from "OPEN"       [2021-07-23 Fri 10:09]
:END:

** CLOSED Improve gemeini rendering speed
:LOGBOOK:
- State "CLOSED"     from "OPEN"       [2021-07-31 Sat 00:18]
:END:

Currently pages with many links render extremely slowly.

Example (>2000 links, 15s): gemini://rawtext.club/~sloum/geminilist/

It turns out that by far the main contributor to this is the use of
(url-port) in elpher-address-from-gemini-url.  I encountered this
problem once before in elpher-remove-redundant-ports.  This function
call is just incredibly slow for some bizarre reason.  Happily,
(url-portspec) is functionally equivalent and is orders of magnitude
faster.  With this replacement, loading the above page takes ~2s
and there aren't any other hotspots.