Repository: pulldown-cmark/pulldown-cmark (Rust)

Issue:
Option to insert reference link definitions on demand
It would be very helpful to be able to register a callback function that is called whenever the parser encounters a reference link it doesn't have a definition for.

If I'm not mistaken, it should suffice to add a new constructor `Parse::with_ref_link_callback(text, fn)` and call `fn`, if set, here instead of returning `None` directly:

https://github.com/google/pulldown-cmark/blob/c1ac9123f1653dc774403634696f848ffecbb01c/src/parse.rs#L1301

Please let me know if this is something you want to do in this project, and what your preferred API would look like.

## Context

@QuietMisdreavus and @Manishearth are currently working on implement [Rust RFC 1946](https://github.com/rust-lang/rfcs/blob/b9b71bbddd7ae54b338e3c17f47f30b9602cc338/text/1946-intra-rustdoc-links.md) "Intra rustdoc links" in <https://github.com/rust-lang/rust/pull/47046>.

The RFC's goal is to make it easy to link to Rust items in doc comments, and the main way to do that is by adding "implied ref links": You can write a doc comment containing

> This calls [foo()] with the given [::bar::Baz].

and rustdoc will generate the correct links to the function `foo` and the type `Baz`.

Rustdoc will soon use pulldown-cmark, and our goal is to land this feature once it does. Sadly, the current API of pulldown will return links without definitions as text events, so we can't dynamically generate and add these links. Adding all link definitions up front is not viable, as there may be links to external crates, which means the number of possible definitions is infinite.


Discussion:
Yes, I've thought about adding a feature like this, and the API sketch you propose looks good. If the API has been considered by the rustdoc people I will very likely be ok with adding it.

I'm a bit overloaded with other things right now, but if adding this is blocking on me, feel free to ping me.
The api seems like it would fit well in rustdoc. I can do the engineering
work for this, shouldn't be hard.

We're not blocked on this right now since we need to completely finish
swapping out the old parser before we introduce pulldown-only features. So
no rush.

On Jan 8, 2018 10:50 PM, "Raph Levien" <notifications@github.com> wrote:

> Yes, I've thought about adding a feature like this, and the API sketch you
> propose looks good. If the API has been considered by the rustdoc people I
> will very likely be ok with adding it.
>
> I'm a bit overloaded with other things right now, but if adding this is
> blocking on me, feel free to ping me.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/google/pulldown-cmark/issues/121#issuecomment-356032672>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABivSPPYe9EyeeMrfEG0JJqUMXoHjlFoks5tIk5WgaJpZM4RU-qU>
> .
>

https://github.com/google/pulldown-cmark/pull/123



Patch:
diff --git a/src/parse.rs b/src/parse.rs
index eac8c799..040cb3af 100644
--- a/src/parse.rs
+++ b/src/parse.rs
@@ -64,6 +64,10 @@ pub struct RawParser<'a> {
     containers: Vec<Container>,
     last_line_was_empty: bool,
 
+    /// In case we have a broken link/image reference, we can call this callback
+    /// with the reference name and use the link/title pair returned instead
+    broken_link_callback: Option<&'a Fn(&str) -> Option<(String, String)>>,
+
     // state for code fences
     fence_char: u8,
     fence_count: usize,
@@ -145,8 +149,10 @@ bitflags! {
 const MAX_LINK_NEST: usize = 10;
 
 impl<'a> RawParser<'a> {
-    pub fn new_with_links(text: &'a str, opts: Options,
-            links: HashMap<String, (Cow<'a, str>, Cow<'a, str>)>) -> RawParser<'a> {
+    pub fn new_with_links_and_callback(text: &'a str, opts: Options,
+            links: HashMap<String, (Cow<'a, str>, Cow<'a, str>)>,
+            callback: Option<&'a Fn(&str) -> Option<(String, String)>>)
+    -> RawParser<'a> {
         let mut ret = RawParser {
             text: text,
             off: if text.starts_with("\u{FEFF}") { 3 } else { 0 },
@@ -162,6 +168,8 @@ impl<'a> RawParser<'a> {
             fence_count: 0,
             fence_indent: 0,
 
+            broken_link_callback: callback,
+
             // info, used in second pass
             loose_lists: HashSet::new(),
             links: links,
@@ -171,8 +179,13 @@ impl<'a> RawParser<'a> {
         ret
     }
 
+    pub fn new_with_links(text: &'a str, opts: Options,
+            links: HashMap<String, (Cow<'a, str>, Cow<'a, str>)>) -> RawParser<'a> {
+        Self::new_with_links_and_callback(text, opts, links, None)
+    }
+
     pub fn new(text: &'a str, opts: Options) -> RawParser<'a> {
-        RawParser::new_with_links(text, opts, HashMap::new())
+        Self::new_with_links(text, opts, HashMap::new())
     }
 
     // offset into text representing current parse position, hopefully
@@ -1298,7 +1311,17 @@ impl<'a> RawParser<'a> {
             let reference = self.normalize_link_ref(&data[ref_beg..ref_end]);
             let (dest, title) = match self.links.get(&reference) {
                 Some(&(ref dest, ref title)) => (dest.clone(), title.clone()),
-                None => return None
+                None => {
+                    if let Some(ref callback) = self.broken_link_callback {
+                        if let Some(val) = callback(&reference) {
+                            (val.0.into(), val.1.into())
+                        } else {
+                            return None;
+                        }
+                    } else {
+                        return None;
+                    }
+                }
             };
             (dest, title, text_beg, text_end, i)
         };
diff --git a/src/passes.rs b/src/passes.rs
index 43848c5b..7f393cdf 100644
--- a/src/passes.rs
+++ b/src/passes.rs
@@ -33,7 +33,18 @@ impl<'a> Parser<'a> {
     pub fn new(text: &'a str) -> Parser<'a> {
         Parser::new_ext(text, Options::empty())
     }
-    pub fn new_ext(text: &'a str, mut opts: Options) -> Parser<'a> {
+    pub fn new_ext(text: &'a str, opts: Options) -> Parser<'a> {
+        Self::new_with_broken_link_callback(text, opts, None)
+    }
+
+    /// In case the parser encounters any potential links that have a broken
+    /// reference (e.g `[foo]` when there is no `[foo]: ` entry at the bottom)
+    /// the provided callback will be called with the reference name,
+    /// and the returned pair will be used as the link name and title if not
+    /// None.
+    pub fn new_with_broken_link_callback(text: &'a str, mut opts: Options,
+            callback: Option<&'a Fn(&str) -> Option<(String, String)>>)
+            -> Parser<'a>  {
         opts.remove(OPTION_FIRST_PASS);
         // first pass, collecting info
         let first_opts = opts | OPTION_FIRST_PASS;
@@ -43,7 +54,7 @@ impl<'a> Parser<'a> {
 
         // second pass
         let loose_lists = info.loose_lists;
-        let second = RawParser::new_with_links(text, opts, info.links);
+        let second = RawParser::new_with_links_and_callback(text, opts, info.links, callback);
 
         //println!("loose lists: {:?}", info.loose_lists);
         Parser {


Test patch:
diff --git a/tests/html.rs b/tests/html.rs
index 58ff9f9e..c7d945cb 100644
--- a/tests/html.rs
+++ b/tests/html.rs
@@ -206,3 +206,35 @@ console.log("fooooo");
 
     assert_eq!(expected, s);
 }
+
+#[test]
+fn html_test_broken_callback() {
+    let original = r##"[foo],
+[bar],
+[baz],
+
+   [baz]: https://example.org
+"##;
+
+    let expected = r##"<p><a href="https://replaced.example.org" title="some title">foo</a>,
+[bar],
+<a href="https://example.org">baz</a>,</p>
+"##;
+
+    use pulldown_cmark::{Options, Parser, html};
+
+    let mut s = String::new();
+
+    let callback = |reference: &str| -> Option<(String, String)> {
+        if reference == "foo" || reference == "baz" {
+            Some(("https://replaced.example.org".into(), "some title".into()))
+        } else {
+            None
+        }
+    };
+
+    let p = Parser::new_with_broken_link_callback(&original, Options::empty(), Some(&callback));
+    html::push_html(&mut s, p);
+
+    assert_eq!(expected, s);
+}