drm: Always reject drm_vblank_get() after drm_vblank_off()

Make sure drm_vblank_get() never succeeds when called between
drm_vblank_off() and drm_vblank_on(). Borrow a trick from the
old drm_vblank_{pre,post}_modeset() functions and just bump
the refcount in drm_vblank_off() and drop it in drm_vblank_on().

When drm_vblank_get() encounters a >0 refcount and the vblank
interrupt is already disabled it will simply return -EINVAL.

Hopefully the use of inmodeset won't conflict badly with
drm_vblank_{pre,post}_modeset().

For i915 there's a window between drm_vblank_off() and marking the
crtc as inactive where the current code still allows drm_vblank_get().

v2: Describe what drm_vblank_get() does to explain how
    a simple refcount bump manages to fix things (Daniel)

Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
drm/i915: Warn if drm_vblank_get() still works after drm_vblank_off()

v2: Drop the drm_vblank_off() (Daniel)
    Use drm_crtc_vblank_{get,put}()

Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
drm: Don't clear vblank timestamps when vblank interrupt is disabled

Clearing the timestamps causes us to send zeroed timestamps to userspace
if they get sent out in response to the drm_vblank_off(). It's better
to send the very latest timestamp and count instead.

Testcase: igt/kms_flip/modeset-vs-vblank-race
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
drm: Move drm_update_vblank_count()

Move drm_update_vblank_count() to avoid forward a declaration.
No functional change.

Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
drm: Have the vblank counter account for the time between vblank irq disable and drm_vblank_off()

If the vblank irq has already been disabled (via the disable timer) when
we call drm_vblank_off() sample the counter and timestamp one last time.
This will make the sure that the user space visible counter will account
for time between vblank irq disable and drm_vblank_off().

Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
drm: Avoid random vblank counter jumps if the hardware counter has been reset

When drm_vblank_on() is called the hardware vblank counter may have
been reset, so we can't trust that the old values sampled prior to
drm_vblank_off() have anything to do with the new values.

So update the .last count in drm_vblank_on() to make the first
drm_vblank_enable() consider that as the reference point. This
will correct the user space visible counter to account for the
time between drm_vblank_on() and the first drm_vblank_enable()
calls.

For extra safety subtract one from the .last count in drm_vblank_on()
to make sure that user space will never see the same counter value
before and after modeset.

Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
drm: Reduce the amount of dev->vblank[crtc] in the code

Declare a local struct drm_vblank_crtc * and use that
instead of having to do dig it out via 'dev->vblank[crtc]'
everywhere.

Performed with the following coccinelle incantation,
and a few manual whitespace cleanups:

@@
identifier func,member;
expression num_crtcs;
struct drm_device *dev;
unsigned int crtc;
@@
func (...) {
+ struct drm_vblank_crtc *vblank;
...
if (crtc >= num_crtcs)
   return ...;
+ vblank = &dev->vblank[crtc];
<+...
(
- dev->vblank[crtc].member
+ vblank->member
|
- &(dev->vblank[crtc])
+ vblank
)
...+>
}

@@
struct drm_device *dev;
int crtc;
identifier member;
expression num_crtcs;
@@
for (crtc = 0; crtc < num_crtcs; crtc++) {
+ struct drm_vblank_crtc *vblank = &dev->vblank[crtc];
+
<+...
(
- dev->vblank[crtc].member
+ vblank->member
|
- &(dev->vblank[crtc])
+ vblank
)
...+>
}

@@
identifier func,member;
@@
func (struct drm_device *dev, int crtc, ...) {
+ struct drm_vblank_crtc *vblank = &dev->vblank[crtc];
<+...
(
- dev->vblank[crtc].member
+ vblank->member
|
- &(dev->vblank[crtc])
+ vblank
)
...+>
}

v2: Rebased

Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
drm: Fix deadlock between event_lock and vbl_lock/vblank_time_lock

Currently both drm_irq.c and several drivers call drm_vblank_put()
while holding event_lock. Now that drm_vblank_put() can disable the
vblank interrupt directly it may need to grab vbl_lock and
vblank_time_lock. That causes deadlocks since we take the locks
in the opposite order in two places in drm_irq.c. So let's make
sure the locking order is always event_lock->vbl_lock->vblank_time_lock.

In drm_vblank_off() pull up event_lock from underneath vbl_lock. Hold
the event_lock across the whole operation to make sure we only send
out the events that were on the queue when we disabled the interrupt,
and not ones that got added just after (assuming drm_vblank_on() already
managed to get called somewhere between).

To sort the other deadlock pull the event_lock out from
drm_handle_vblank_events() into drm_handle_vblank() to be taken outside
vblank_time_lock. Add the appropriate assert_spin_locked() to
drm_handle_vblank_events().

Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
drm: Fix race between drm_vblank_off() and drm_queue_vblank_event()

Currently it's possible that the following will happen:
1. drm_wait_vblank() calls drm_vblank_get()
2. drm_vblank_off() gets called
3. drm_wait_vblank() calls drm_queue_vblank_event() which
   adds the event to the queue event though vblank interrupts
   are currently disabled (and may not be re-enabled ever again).

To fix the problem, add another vblank->enabled check into
drm_queue_vblank_event().

drm_vblank_off() holds event_lock around the vblank disable,
so no further locking needs to be added to drm_queue_vblank_event().
vblank disable from another source is not possible since
drm_wait_vblank() already holds a vblank reference.

Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
drm: Disable vblank interrupt immediately when drm_vblank_offdelay<0

Make drm_vblank_put() disable the vblank interrupt immediately when the
refcount drops to zero and drm_vblank_offdelay<0.

v2: Preserve the current drm_vblank_offdelay==0 'never disable' behaviur

Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
drm: Add dev->vblank_disable_immediate flag

Add a flag to drm_device which will cause the vblank code to bypass the
disable timer and always disable the vblank interrupt immediately when
the last reference is dropped.

v2: Add some notes about the flag to the kernel doc

Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
drm/i915: Opt out of vblank disable timer on >gen2

Now that the vblank races are plugged, we can opt out of using
the vblank disable timer and just let vblank interrupts get
disabled immediately when the last reference is dropped.

Gen2 is the exception since it has no hardware frame counter.

Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
drm: Kick start vblank interrupts at drm_vblank_on()

If the user is interested in getting accurate vblank sequence
numbers all the time they may disable the vblank disable timer
entirely. In that case it seems appropriate to kick start the
vblank interrupts already from drm_vblank_on().

v2: Adapt to the drm_vblank_offdelay ==0 vs <0 changes

Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
drm/i915: Update scanline_offset only for active crtcs

update_scanline_offset() in intel_sanitize_crtc() was supposed to
be called only for active crtcs. But due to some underrun patches it
now gets updated for all crtcs on gmch platforms.

Move the update_scanline_offset() to the very beginning of
intel_sanitize_crtc() where we update the vblank state. This seems like
a better place anyway since the scanline offset ought to be up to date
before we might need to consult it. So before any vblanky stuff happens.

Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
drm: Fix confusing debug message in drm_update_vblank_count()

Now that drm_update_vblank_count() can be called even when we're not
about to enable the vblank interrupts we shouldn't print debug messages
stating otherwise.

Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
drm: Store the vblank timestamp when adjusting the counter during disable

During vblank disable the code tries to guess based on the
timestamps whether we just missed one vblank or not. And if so
it increments the counter. However it forgets to store the new
timestamp to the approriate slot in our timestamp ring buffer.
So anyone querying the timestamp for the resulting sequence
number would get a stale timestamp. Fix it up by storing the
new timestamp.

Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
drm: Remove drm_vblank_cleanup from drm_vblank_init error path.

drm_vblank_cleanup() would operate on non-existent dev->vblank
data structure, as failure to allocate that data structure is
what triggers the error path in the first place.

Signed-off-by: Mario Kleiner <mario.kleiner.de@gmail.com>
Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
gpu: ipu-v3: Add ipu-cpmem unit

Move channel parameter memory setup functions and macros into a new
submodule ipu-cpmem. In the process, cleanup arguments to the functions
to take a channel pointer instead of a pointer into cpmem for that
channel. That allows the structure of the parameter memory to be
private to ipu-cpmem.c.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
staging: imx-drm: Convert to new ipu_cpmem API

The ipu_cpmem_*() calls now take a channel pointer instead of a
pointer into cpmem for that channel.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
gpu: ipu-v3: Add functions to set CSI/IC source muxes

Adds two new functions, ipu_set_csi_src_mux() and ipu_set_ic_src_mux(),
that select the inputs to the CSI and IC respectively. Both muxes are
programmed in the IPU_CONF register.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>