WEB: Adding new pandas website
State: closed

The website can be seen here: https://datapythonista.github.io/pandas-web/

It's still pending that the designer polish the styles. And also the content can surely be improved with everybody's feedback. But I think will be useful to merge this once we iterate a bit on people's feedback, so we can start working on integrating the docs.

I'm coordinating with the devs of numpy/scipy, scikit-learn and matplotlib to see if we can reshare things implemented here. Ideally I'd like to have the `pysuerga.py` script in a separate repo (that's why it's not named pandas_web.py`), and also the layout and pages that can be shared (the code of conduct, the donate, the blog,...). For now I add everything here for simplicity, and will create a new project and have it as a dependency here if they are interested.

CC: @pandas-dev/pandas-core 

Labels:
name: Web
description: pandas website

Comments:
Looks good. Very clean layout.

A few comments:
1: The links to the docs don't work. That's perhaps in the works?

2: Is TideShift a Sponsor of Pandas now? Then they should be on the "with the support of..." list?

3: The link to Wes McKinney's book is http://www.kqzyfj.com/click-7040302-11260198?url=http%3A%2F%2Fshop.oreilly.com%2Fproduct%2F0636920050896.do&cjsku=0636920023784. Is there a reason for going through that ``http://www.kqzyfj.com`` link and not directly to oreilly.com? Looks a bit fishy going through redirects for web shopping, IMO.

Thanks for the review @topper-123, good feedback. I added Tidelift to the list of partners, and the docs links not working is intentional. I'm planning to add them after this is merged, and I make the docs use this same layout. Now they'd simply link to the current docs, and you'd leave the website.

Regarding Wes book, I guess that redirect was to keep track on the number of clicks. But besides looking fishy, clicking on it asks me to confirm that I accept some weird conditions, and after that it takes me to the home page of O'Reilly, not the book page (I guess that's because the 2nd edition has been released)_.

I updated the link to the direct link to the second edition, and also change the image to be the second edition front page. @wesm if you want to track clicks to the book, I think the best is to create a page in our own website that performs an automatic redirect, and track the visits with Google Analytics. Let me know if you want me to implement that.
From a mobile perspective,  most everything rendered okay.

One consistent thing I noticed was that text was a _little_ small for all pages:

![Screenshot_20190819-181603](https://user-images.githubusercontent.com/10647082/63369224-57302700-c334-11e9-9159-317b691cd406.png)

Also this page didn't render too well:

![Screenshot_20190819-181430](https://user-images.githubusercontent.com/10647082/63369351-99596880-c334-11e9-862c-da47a6836057.png)


Wouldn't say these are huge priorities, but keeping it on the radar.
Thanks for the feedback @mroeschke. I'll let the designer from Indeed take care of that. The website uses bootstrap, and should be very easy to make it responsive and look perfect in mobile. But I don't know exactly how it works, so I'll let her do it.
Removed the carousel, and made couple of changes to reduce the size of the website files. There are couple of things I want to change to reduce the size further. After that, if there is no particular change requested, I think we should merge and continue with follow ups. This way we can work in parallel with the integration with the docs and the CI to display the dev version of the web.

The pending changes are:
- Change Anaconda logo to a svg version (there are eps files that can be converted in https://www.anaconda.com/media-kit/)
- Convert to jpeg and increase the compression of the files in the installation page. If anyone wants to propose changes to that page, now it's a good time, before we increase the size of the repo with unnecessary files.
I made the web responsive (couple of things still need to be changed to look perfect in mobile, but almost there).

Also removed the "try pandas" page for now and the blog (will focus on the core here, and add those as a follow up).

The version in https://datapythonista.github.io/pandas-web/ is almost the one I'd like to get merged here, and then work from there in other PRs. Any feedback welcome.
Glad to hear it.

Roughly speaking, what's the total size of the new files / assets here?
Just glanced briefly from phone where it looks really good. Just as a heads up though I think all of the doc pages are broken - at least navigating through gives a 404

https://datapythonista.github.io/pandas-web/docs/user_guide/

Sorry if I missed this on the call yesterday but how does this fit into what Stijn is doing? Would you want this merged as a precursor or does it replace some aspects?

Sent from my iPhone

> On Sep 12, 2019, at 9:43 AM, Tom Augspurger <notifications@github.com> wrote:
> 
> Glad to hear it.
> 
> Roughly speaking, what's the total size of the new files / assets here?
> 
> —
> You are receiving this because you are on a team that was mentioned.
> Reply to this email directly, view it on GitHub, or mute the thread.

I think these two can proceed mostly independently.

Marc's is updating the website. Stijn's is updating the docs.

We'll need to reconcile the styles by using a single CSS file for both, but
that can be done afterwards I think.

On Thu, Sep 12, 2019 at 11:46 AM William Ayd <notifications@github.com>
wrote:

> Just glanced briefly from phone where it looks really good. Just as a
> heads up though I think all of the doc pages are broken - at least
> navigating through gives a 404
>
> https://datapythonista.github.io/pandas-web/docs/user_guide/
>
> Sorry if I missed this on the call yesterday but how does this fit into
> what Stijn is doing? Would you want this merged as a precursor or does it
> replace some aspects?
>
> Sent from my iPhone
>
> > On Sep 12, 2019, at 9:43 AM, Tom Augspurger <notifications@github.com>
> wrote:
> >
> > Glad to hear it.
> >
> > Roughly speaking, what's the total size of the new files / assets here?
> >
> > —
> > You are receiving this because you are on a team that was mentioned.
> > Reply to this email directly, view it on GitHub, or mute the thread.
>
> —
> You are receiving this because you are on a team that was mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/28014?email_source=notifications&email_token=AAKAOIU44MQQLID6UEJVLHLQJJW7XA5CNFSM4INCLXP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6SQXEQ#issuecomment-530910098>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAKAOIX6D3HK3FGLZKRM4Z3QJJW7XANCNFSM4INCLXPQ>
> .
>

Regarding size, I think it's around 500kb, but I still want to optimize the images in the Install page (the rest is almost nothing).

The 404 for the docs are expected. The plan is to merge this and then start integrating both, so there is no perception from the user that they are built separately. There are some challenges there, but spoke with Joris at EuroSciPy, and seems doable.
There is also an issue with sizing of content on the Contributors page due to long user names. Maybe OK to just have names and link to GH site if people wanted

<img width="745" alt="image" src="https://user-images.githubusercontent.com/609873/64804628-f87d5800-d543-11e9-9326-9156cf27a42b.png">

> We'll need to reconcile the styles by using a single CSS file for both, but that can be done afterwards I think.

Sounds good, though I'm wondering if we should commit to one style as part of this change then. Seems like it will be more hassle in the long run if the docs and rest of the site are built with two different style templates in mind and then we try merging later
> Seems like it will be more hassle in the long run if the docs and rest of the site are built with two different style templates in mind and then we try merging later

Yeah, I haven't looked, but if each is using a lot of custom CSS then merging later will be a pain. I think on the call yesterday Joris and Marc both said they're basically just using bootstrap's defaults?
> > Seems like it will be more hassle in the long run if the docs and rest of the site are built with two different style templates in mind and then we try merging later
>
> Yeah, I haven't looked, but if each is using a lot of custom CSS then merging later will be a pain. I think on the call yesterday Joris and Marc both said they're basically just using bootstrap's defaults?

Yes, this is still a technicality we need to figure out how to combine both in the easiest way.

Note that I personally think they can still have a somewhat distinct look (this also will have a different layout, for example), but just nicely integrated compared to the 2 completely separate entities as we have now (eg pytorch.org has separate layout for main website pages and docs, but a common header and of course also common typology and styling)

Currently Marc is not yet using much css, so that should not create too much conflicts. For the doc theme we have some more css as we have to style the sidebars (which is not a completely standard component of bootstrap) and we have to style the sphinx-specific html constructs.
Made all the pending updates. The changes add 324Kb to the repo, I don't think this can be reduced more (I reduced the size of every image where it made sense).

I added a temporary hack to visualize how the logos look like:
- http://datapythonista.github.io/pandas-web/#dataframe
- http://datapythonista.github.io/pandas-web/#panda
- http://datapythonista.github.io/pandas-web/#lines

Also, if you want to try a different logo, you can provide the url to the image after the hash: http://datapythonista.github.io/pandas-web/#https://kunlabori.es/wp-content/uploads/2014/12/python-madrid.png

I addressed most of the reported issues, and also improved few styles, thanks all for the feedback.

Comments on the remaining topics:

- **Governance**: I don't see what's the advantage of a separate repo. The governance content here is partially generated from the info in `config.yml`, which is also reused in other parts. I do see a clear advantage of having the info just in one place, so it's not outdated as it happens often with the governance repo. What is the advantage of a separate repo?

- **Development/contributing docs**: The reason why I moved it to the web is because we're not interested in the version of the last release (e.g. `0.25.1`) but instead the latest (`master`). Having it in the website makes this happen automatically. Having it in the docs requires some dirty hacks I think. That's also the reason why I finally discontinued the idea of building the whole website with Sphinx. How would you address this if you have it in the docs, do you have a solution?

- **Custom static site generator**: Your guess is right, I think the advantage of not using Pelican or Hugo is simplicity and flexibility. The static site generator part is around 120 lines of code (the rest are things like fetching the maintainers data from github, which we need anyway). I like Pelican, but besides the added complexity it has limitations that sometimes require complicating things significantly when trying to do simple things. I think `pandas_web.py` is very simple, and adapted exactly to what we need (I don't think we need to add anything else).

- **Home page, footer, about us, github link...**: All those sound good, but I think they make more sense in follow ups. If there can be a quick agreement in any of them I'm happy to implement it here. But surely the home page, and probably the others, they'll be better addressed with an independent PR, so we can have a focused discussion.
On the topic of a separate repo and a custom site generator do we know what this would add in terms of complexity to our documented build process? I think the current site is relatively simple and generated via make rules with how's its set up in a separate repo, so would ideally like to have a drop in replacement for that generation if not already included
Not sure I understand you @WillAyd. The generation of the website takes around 7 seconds (most of the time is spent fetching the maintainers data from github). It just adds a `./pandas_web.py pandas` to the process.

I think in terms of complexity this is simpler than using Pelican, and adding a make file here would make things more complex. Also, I think having a directory web instead of a separate repo make things simpler (and I'd say it'd help avoid that we forget to update things in the separate website, which happened some times with the current website).

Can you please clarify what you mean, not sure what you're proposing.
@jorisvandenbossche probably worth having a look at this when you have time, and see if we can merge the website soon (even a minimal version), so we can work on the integration of the web and docs, and then continue from there. I think the earlier we have the merge the better, so we don't risk having to change too much stuff later on.
What are our thoughts on when pandas.pydata.org will be switched over? Coincident with the 1.0 release?
> What are our thoughts on when pandas.pydata.org will be switched over? Coincident with the 1.0 release?

Coincident with 1.0 sounds good to me. Also ok to publish the new website earlier when it's ready, if people prefer that.
We'll certainly want to continuously deploy somewhere, for ease of
inspecting changes.

But announcing it to the world with 1.0 sounds good.

On Mon, Sep 16, 2019 at 12:01 PM Marc Garcia <notifications@github.com>
wrote:

> What are our thoughts on when pandas.pydata.org will be switched over?
> Coincident with the 1.0 release?
>
> Coincident with 1.0 sounds good to me. Also ok to publish the new website
> earlier when it's ready, if people prefer that.
>
> —
> You are receiving this because you are on a team that was mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/28014?email_source=notifications&email_token=AAKAOIT7QDTC5FE4L5JZHB3QJ63XTA5CNFSM4INCLXP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6Z2FHA#issuecomment-531866268>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAKAOIR7XI2BY6QZSRO5FPLQJ63XTANCNFSM4INCLXPQ>
> .
>

@pandas-dev/pandas-core let me know if there is any other comment here, otherwise I'll merge in the next hours, so I can move forward with the rest of the tasks for the new website (add build to the CI, integrate with docs, open discussion about the home page...)
For the governance document and contributing docs, I would suggest to leave them out for now, and add them back in a PR, so we can have a focused discussion on those in a separate PR. Otherwise I want to have that discussion here.

When merging this, we should not have duplicated content, I think


I am trying it out locally, but the images are not working for me. Changing `/static/...` to `static/..` in the paths (eg on the main page) seems to fix this, but not sure it is because I am doing something wrong.

Also the links in the navbar don't work locally. 
Is this maybe because I am just opening the html files in the browser (as I am used to with sphinx), and I rather need to serve them on a localhost?

--

Can you add a README in the `web/` directory shortly stating the purpose of that directory?
Addressed comments.

You need to run `cd build && python -m http.server` to navigate the website, links and image paths are absolute, not relative. If you want to build the website to live in a directory, you need the `--base-url=/directory-for-the-web`.
> You need to run cd build && python -m http.server to navigate the website ...

Thanks! Can you add that note to the readme / command help?


Another thing: the salsalabs donate container is not working (not locally, but also not on your hosted version)
The iframe works for me, not sure if it was a temporary problem in their side, or you've got an adblocker or something preventing it to render.
I didn't want to include the logo, to avoid increasing the size of the repo with something will be replaced soon. I'm removing the logo from the html for now, so it doesn't show broken.
> I didn't want to include the logo, to avoid increasing the size of the repo with something will be replaced soon. I'm removing the logo from the html for now, so it doesn't show broken.

Linking to the logo file in the docs does not work? (and I suppose on build copying that to the build directory)
That logo has the font in black, I could link it, but I'd have to change the header to white to look ok.

I think we still need several iterations with the website until we want to publish it to production, so I would focus on having a starting point so we can open PRs to it, and not in trying to have something final (it's far from it anyway, starting by the home page).