Model: bsmith3715/legal-ft-cert-challenge_final
Author: bsmith3715
Pipeline: sentence-similarity
Library: sentence-transformers
Tags: sentence-transformers, safetensors, bert, sentence-similarity, feature-extraction, generated_from_trainer, dataset_size:196, loss:MatryoshkaLoss, loss:MultipleNegativesRankingLoss, arxiv:1908.10084, arxiv:2205.13147, arxiv:1705.00652, base_model:Snowflake/snowflake-arctic-embed-m, base_model:finetune:Snowflake/snowflake-arctic-embed-m, model-index, text-embeddings-inference, endpoints_compatible, region:us
Base models (finetune): id: Snowflake/snowflake-arctic-embed-m
Card metadata: tags: sentence-transformers, sentence-similarity, feature-extraction, generated_from_trainer, dataset_size:196, loss:MatryoshkaLoss, loss:MultipleNegativesRankingLoss
base_model: Snowflake/snowflake-arctic-embed-m
widget: source_sentence: Who are the authors mentioned in the Q1 2025 tidymodels digest?
sentences: For now, all of these packages only live on my GitHub profile. In the coming weeks, I plan to revisit each of them, squash a bunch of bugs, and send them off to CRAN.
That said, these packages are very much experimental. The user interface of writing directly to users’ files very much limits how useful these tools can be, and I think that the kinds of improvements to interface I’m hoping for may only be possible via some backend other than the RStudio API. I’m looking forward to seeing what that could look like.




Pronounced “c-l-i pal.” ↩︎


In reality, there are bugs and differences here and there, but the development effort to get these packages working in Positron was relatively minimal. ↩︎, rlang::abort() to 
cli::cli_abort():
# before:
rlang::abort("`save_pred` can only be used if the initial results saved predictions."), Q1 2025 tidymodels digest
































Tidyverse




Packages
Blog
Learn
Help
Contribute









Q1 2025 tidymodels digest




Photo by George Stagg and Max Kuhn



  2025/02/27
      

 
      
          
            tidymodels

 
        




  

  Max Kuhn, source_sentence: Who contributed significantly to the development of the new modeling mode of "quantile regression" in parsnip?
sentences: Organizing Tidy Dev Day issues.

I love these sessions for many reasons, but mostly because we meet users and contributors to our packages in person and work with them on specific tasks.
There is a lot more to do here; we have a lot of secondary packages that would benefit from these improvements too.
Quantile regression in parsnip
  





One big update in parsnip was a new modeling mode of "quantile regression". Daniel McDonald and Ryan Tibshirani largely provided some inertia for this work based on their 
disease modeling framework.
You can generate quantile predictions by first creating a model specification, which includes the quantiles that you want to predict:
library(tidymodels)
tidymodels_prefer(), clubs

The operative word here being “want”. Once again, someone with enough experience with R and the tidyverse would not explicitly ask for these improvements; they would just make them themselves and carry on. But someone who is just learning the tidyverse would likely stop much earlier. In my experience, many new learners stop after the very first prompt, which looks unfamiliar, and hence potentially intimidating. For learners of the tidyverse, the simple “Can you do this with the tidyverse?” can be a very useful prompt improvement that gets them majority of the way there.
Case study 3: Web scraping, Since these packages are written in R, they have access to your R environment. This is quite the differentiator compared to the more language-agnostic tools out there—these packages can see the data frames you have loaded, the columns and column types in them, etc. When working with other tools for LLM code-assist that don’t have this information, the friction of printing out variable information from my R environment and pasting it into whatever interface is so high that I don’t even ask LLMs for help with tasks they’re otherwise totally capable of., source_sentence: How does the `if_else` function determine the value of `membership_status` in the context of the `clubs` tibble?
sentences: # Load the billboard dataset
data("billboard")

# Prepare the data for visualization
billboard_long <- billboard |>
  pivot_longer(
    cols = starts_with("wk"),
    names_to = "week",
    values_to = "rank",
    names_prefix = "wk",
    values_drop_na = TRUE
  )

# Convert week to a numeric value
billboard_long$week <- as.numeric(billboard_long$week)

# Create the visualization
ggplot(billboard_long, aes(x = week, y = rank, group = track, color = track)) +
  geom_line(show.legend = FALSE, alpha = 0.6) +
  scale_y_reverse() +  # Reverse the y-axis since rank 1 is the top position
  labs(
    title = "Song Rank vs. Week Number",
    x = "Week Number",
    y = "Rank",
    caption = "Data: Billboard (tidyr package)"
  ) +
  theme_minimal(), Once again, it’s possible to ask the tool to make these updates. At the time of developing this example, Claude didn’t provide a sharing link for the conversation (it does now!), but I’ve saved it at https://gist.github.com/mine-cetinkaya-rundel/c74dc7c5f2a27a846574edd063988431. Even with vague prompts like “If clubs is a tibble, it’ll only print out the first 10 rows anyway,” it does exactly what we would want it to do, resulting in the following:

clubs <- clubs |>
  mutate(membership_status = if_else(is.na(membership_status), "Closed", "Open")) |>
  relocate(membership_status)

clubs, When triggered, read what’s inside of the selection using rstudioapi.
Pass that selection contents to an LLM along with a system prompt describing how to convert R erroring code to use cli using ellmer. (If you’re curious, the current draft of that prompt is 
here.)
When the response is returned, replace the contents of the selection with the response using cli.

This approach of using ellmer and the rstudioapi has its ups and downs. As for the advantages:, source_sentence: What is meant by "code smell" in the context of programming?
sentences: clipal was a huge boost for us in the most recent spring cleaning. Depending on the code being updated, these erroring calls used to take 30 seconds to a few minutes. With clipal, though, the model could usually get the updated code 80% or 90% of the way there in a couple seconds. Up to this point, irritated by autocomplete and frustrated by the friction of copying and pasting code and typing out the same bits of context into chats again and again, I had been relatively skeptical that LLMs could make me more productive. After using clipal for a week, though, I began to understand how seamlessly LLMs could automate the cumbersome and uninteresting parts of my work., Improve code smell: While I don’t have empirical evidence for this, I believe for humans, taste for good code develops faster than ability. For LLMs, this is the opposite. These tools will happily barf out code that runs without regard to cohesive syntax, avoiding redundancies, etc. Therefore, it’s essential to “clean up” the suggested code to improve its “code smell”. Below are some steps I regularly use:, Organizing Tidy Dev Day issues.

I love these sessions for many reasons, but mostly because we meet users and contributors to our packages in person and work with them on specific tasks.
There is a lot more to do here; we have a lot of secondary packages that would benefit from these improvements too.
Quantile regression in parsnip
  





One big update in parsnip was a new modeling mode of "quantile regression". Daniel McDonald and Ryan Tibshirani largely provided some inertia for this work based on their 
disease modeling framework.
You can generate quantile predictions by first creating a model specification, which includes the quantiles that you want to predict:
library(tidymodels)
tidymodels_prefer(), source_sentence: How did users interact with the clipal1 R package to update their code?
sentences: clipal1, a (now-superseded) R package that allows users to select erroring code, press a keyboard shortcut, wait a moment, and watch the updated code be inlined in to the selection., low_temp <- bind_rows(
  all_tables[[3]] %>% select(Month = 1, Low = 2),
  all_tables[[4]] %>% select(Month = 1, Low = 2)
) %>% mutate(Low = as.numeric(gsub("°F", "", Low)))
#> Error in all_tables[[3]]: subscript out of bounds

# Combine and format final dataset
climate_data <- high_temp %>% 
  left_join(low_temp, by = "Month") %>% 
  mutate(Month = factor(Month, levels = month.name)) %>% 
  arrange(Month) %>% 
  select(Month, High, Low)

# Display formatted results
print(climate_data)
#> # A tibble: 3 × 3
#>   Month  High   Low
#>   <fct> <dbl> <dbl>
#> 1 NA    49    89   
#> 2 NA    28    70   
#> 3 NA     4.45  3.94, As for the disadvantages, there are all sorts of UI bummers about this approach. Above all, these packages write directly to your files. This is great in that it removes the need to copy and paste, and when the model’s response is spot on, it’s awesome. At the same time, if the model starts rambling in an .R file or you want to confirm some difference between your previous code and the new code, the fact that these packages just write right into your files can be a bit annoying. Many other inline LLM code-assist tools out there are based on diffs—they show you proposed changes and some UI element that allows you to accept them, reject them, or ask for revisions. This requires one more step between asking for an LLM to do something and the
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics: cosine_accuracy@1, cosine_accuracy@3, cosine_accuracy@5, cosine_accuracy@10, cosine_precision@1, cosine_precision@3, cosine_precision@5, cosine_precision@10, cosine_recall@1, cosine_recall@3, cosine_recall@5, cosine_recall@10, cosine_ndcg@10, cosine_mrr@10, cosine_map@100
model-index: name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-m
results: task: type: information-retrieval
name: Information Retrieval
dataset: name: Unknown
type: unknown
metrics: type: cosine_accuracy@1
value: 0.8333333333333334
name: Cosine Accuracy@1
verified: no, type: cosine_accuracy@3
value: 0.9166666666666666
name: Cosine Accuracy@3
verified: no, type: cosine_accuracy@5
value: 0.9583333333333334
name: Cosine Accuracy@5
verified: no, type: cosine_accuracy@10
value: 1
name: Cosine Accuracy@10
verified: no, type: cosine_precision@1
value: 0.8333333333333334
name: Cosine Precision@1
verified: no, type: cosine_precision@3
value: 0.3055555555555555
name: Cosine Precision@3
verified: no, type: cosine_precision@5
value: 0.19166666666666668
name: Cosine Precision@5
verified: no, type: cosine_precision@10
value: 0.10000000000000002
name: Cosine Precision@10
verified: no, type: cosine_recall@1
value: 0.8333333333333334
name: Cosine Recall@1
verified: no, type: cosine_recall@3
value: 0.9166666666666666
name: Cosine Recall@3
verified: no, type: cosine_recall@5
value: 0.9583333333333334
name: Cosine Recall@5
verified: no, type: cosine_recall@10
value: 1
name: Cosine Recall@10
verified: no, type: cosine_ndcg@10
value: 0.9168716458952284
name: Cosine Ndcg@10
verified: no, type: cosine_mrr@10
value: 0.8902777777777778
name: Cosine Mrr@10
verified: no, type: cosine_map@100
value: 0.8902777777777778
name: Cosine Map@100
verified: no
Likes: 0 | All-time downloads: 8
Created: 2025-05-13 16:50:29