G EOMETRY MATTERS : E XPLORING LANGUAGE EXAMPLES AT THE DECISION BOUNDARY
Anonymous authors
Paper under double-blind review
A BSTRACT
A growing body of recent evidence has highlighted the limitations of natural language processing (NLP) datasets and classiﬁers.
These include the presence of annotation artifacts in datasets, classiﬁers relying on shallow features like a single word (e.g., if a movie review has the word “romantic”, the review tends to be positive), or unnecessary words (e.g., learning a proper noun to classify a movie as positive or negative).
The presence of such artifacts has subsequently led to the development of challenging datasets to force the model to generalize better.
While a variety of heuristic strategies, such as counterfactual examples and contrast sets, have been proposed, the theoretical justiﬁcation about what makes these examples difﬁcult for the classiﬁer is often lacking or unclear.
In this paper, using tools from information geometry, we propose a theoretical way to quantify the difﬁculty of an example in NLP.
Using our approach, we explore difﬁcult examples for several deep learning architectures.
We discover that both BERT, CNN and fasttext are susceptible to word substitutions in high difﬁculty examples. These classiﬁers tend to perform poorly on the FIM test set.
(generated by sampling and perturbing difﬁcult examples, with accuracy dropping below 50%). We replicate our experiments on 5 NLP datasets (YelpReviewPolarity, AGNEWS, SogouNews, YelpReviewFull and Yahoo Answers).
On YelpReviewPolarity we observe a correlation coefﬁcient of -0.4 between resilience to perturbations and the difﬁculty score. Similarly we observe a correlation of 0.35 between the difﬁculty score and the empirical success probability of random substitutions.
Our approach is simple, architecture agnostic and can be used to study the fragilities of text classiﬁcation models.
All the code used will be made publicly available, including a tool to explore the difﬁcult examples for other datasets.
I NTRODUCTION
Machine learning classiﬁers have achieved state-of-the-art success in tasks such as image classiﬁcation and text classiﬁcation.
Despite their successes, several recent papers have pointed out ﬂaws in the features learned by such classiﬁers.
Geirhos et al.
(2020) cast this phenomenon as shortcut learning, where a classiﬁer ends up relying on shallow features in benchmark datasets that do not generalize well to more difﬁcult datasets or tasks.
For instance, Beery et al.
(2018) showed that an image dataset constructed for animal detection and classiﬁcation failed to generalize to images of animals in new locations.
In language, this problem manifests at the word level.
Poliak et al.
(2018) showed that models using one of the two input sentences for semantic entailment performed better than the majority class by relying on shallow features.
Similar observations were also made by Gururangan et al.
(2018), where linguistic traits such as "vagueness" and "negation" were highly correlated with certain classes.
In order to study the robustness of a classiﬁer, it is essential to perturb the examples at the classiﬁer’s decision boundary.
Contrast sets by Gardner et al.
and counterfactual examples by Kaushik et al.
(2020) are two approaches where the authors aimed at perturbing the datasets to identify difﬁcult examples.
In contrast sets, authors of the dataset manually ﬁll in the examples near the decision boundary (examples highlighted in small circles in Figure 1) to better evaluate the classiﬁer performance.
In counterfactual examples, the authors use counterfactual reasoning along with Amazon
Mechanical Turk to create the "non-gratuitous changes.
" While these approaches are interesting, it’s still unclear if evaluating on these will actually capture a classiﬁer’s fragility.
Furthermore, these approaches signiﬁcantly differ from each other and it’s important to come up with a common way to reason about them.
Motivated by these challenges, we propose a geometrical framework to reason about difﬁcult examples.
Using our method, we are able to discover fragile examples for state of the art NLP models like
BERTby Devlin et al.
and CNN (Convolutional Neural Networks) by Kim (2014).
Our experiments using the Fisher information metric (FIM) show that both counterfactual examples and contrast sets are, in fact, quite far from the decision boundary geometrically and not that different from normal examples (circles and triangles in Figure 1).
As such, it is more important to perform evaluation on the examples lying in the green region, which represent confusing examples for the classiﬁer, where even a small perturbation (for instance, substituting the name of an actress) can cause the neural network to misclassify.
It is important to note that this does not depend solely on the classiﬁer’s certainty as adversarial examples can fool neural networks into misclassifying with high conﬁdence, as was shown by Szegedy et al.
We now motivate our choice of using the Fisher information metric (FIM) in order to quantify the difﬁculty of an example.
In most natural language processing tasks, deep learning models are used to model the conditional probability distribution p ( y | x ) of a class label y conditioned on the input x .
Here x can represent a sentence, while y can be a sentiment of the sentence.
If we imagine a neural network as a probabilistic mapping between inputs to outputs, a natural property to measure is the Kullback-Leibler (KL) divergence between the example and an (cid:15) perturbation around that example.
For small perturbations to the input, the FIM gives a quadratic form that approximates, up to second order, the change in the output probabilities of a neural network.
Zhao et al.
(2019) used this fact to demonstrate that the eigenvector associated with the maximum eigenvalue of the FIM gives an effective direction to perturb an example to generate an adversarial attack in computer vision.
Furthermore, from an information geometry viewpoint, the FIM is a Riemannian metric, inducing a manifold geometry on the input space and providing a notion of distance based on changes in the information of inputs.
To the best of our knowledge, this is the ﬁrst work analyzing properties of the ﬁsher metric to understand classiﬁer fragility in NLP.
The rest of the paper is organized as follows:
In Section 2, we summarize related work.
In Section 3, we discuss our approach of computing the FIM and the gradient-based perturbation strategy.
In
Section 4, we discuss the results of the eigenvalues of FIM in synthetic data and sentiment analysis datasets with BERT and CNN.
Finally, in Section 5, we discuss the implications of studying the eigenvalues of FIM for evaluating NLP models. [Table]
R ELATED W ORK
In NLP, machine learning models for classiﬁcation rely on spurious statistical patterns of the text and use shortcut for learning to classify.
These can range from annotation artifacts, as was shown by Goyal et al.
; Kaushik and Lipton (2018)
; Gururangan et al.
(2018), spelling mistakes as in McCoy et al.
(2019), or new test conditions that require world knowledge Glockner et al.
Simple decision rules that the model relies on are hard to quantify.
Trivial patterns like relying on the answer “2” for answering questions of the format “how many” for the visual question answering dataset Antol et al.
(2015), would correctly answer 39% of the questions.
Jia and Liang (2017) showed that adversarially inserted sentences that did not change the correct answer, would cause state of the art models to regress in performance in the SQuAD Rajpurkar et al.
(2016) question answering dataset.
Glockner et al.
(2018) showed that template-based modiﬁcations by swapping just one word from the training set to create a test set highlighted models’ failure to capture many simple inferences.
Dixon et al.
(2018) evaluated text classiﬁers using a synthetic test set to understand unintended biases and statistical patterns.
Using a standard set of demographic identity terms, the authors reduce the unintended bias without hurting the model performance.
Shen et al. showed that word substitution strategies include stylistic variations that change the sentiment analysis algorithms for similar word pairs.
Evaluations of these models through perturbations of the input sentence are crucial to evaluating the robustness of models.
Another issue of language recently has been that static benchmarks like GLUE by Wang et al.
(2018) tend to saturate quickly because of the availability of ever-increasing compute and harder benchmarks are needed like SuperGlue by Wang et al.
A more sustainable approach to this is the development of moving benchmarks, and one notable initiative in this area is the Adversarial NLI by
Nie et al.
, but most of the research community hardly validate their approach against this sort of moving benchmark.
In the Adversarial NLI dataset, the authors propose an iterative, adversarial human-and-model-in-the-loop solution for Natural Language Understanding dataset collection, where the goal post continuously shifts about useful benchmarks and makes models robust by training the model iteratively on difﬁcult examples.
Approaches like never-ending learning byMitchell et al.
(2018) where models improve, and test sets get difﬁcult over time is critical.
A moving benchmark is necessary since we know that improving performance on a constant test set may not generalize to newly collected datasets under the same condition Recht et al.
; Beery et al.
Therefore, it is essential to ﬁnd difﬁcult examples in a more disciplined way.
Approaches based on geometry have recently started gaining traction in computer vision literature.
Zhao et al.
(2019) et al used a similar approach for understanding adversarial examples in images.
M ETHODS
A neural network with discrete output can be thought of as a mapping between a manifold of inputs to the discrete output space.
Most traditional formulations treat this input space as ﬂat, thus reasoning that the gradient of the likelihood in input space gives us the direction which causes the most signiﬁcant change in terms of likelihood.
However, by imagining the input as a pullback of the output, we obtain a non-linear manifold where the euclidean metric no longer sufﬁces.
A more appropriate choice thus is to use the ﬁsher information as a Riemannian metric tensor.
We ﬁrst introduce the Fisher Metric formulation for language.
For the purposes of the derivation below the following notations are used.
x : Vector of input sentence.
This is an n * d sentence where n is the number of words in the sentence and d is the dimensionality of the word embedding.
y :
Label of class, in our context that is the positive or the negative sentiment.
p(y |x
: The conditional probability distribution between y and x.
KL ( p, q ) :
The KL divergence between distributions p and q for two sentences ∇ f ( x ) :
Gradient of a function
of
f w.r.t x ∇ 2 f ( x ) : Hessian of f(x) w.r.t x
We apply the a perturbation η to modifying a sentence to create a new sentence (eg., a counterfactual example).
We can then see the effect of this perturbation η in terms of change in the probability distribution over labels.
Ideally, we would like to ﬁnd points where a small perturbation can result in a large change in the probability distribution over labels.
[Equation]
We now perform a Taylor expansion of the ﬁrst term on the right hand side
[Equation]
Since the expectation of score is zero and the ﬁrst and last terms cancel out, we are left with.
[Equation]
Where G is the FIM.
By studying the eigenvalues of this matrix locally, we can quantify if small change in η can cause a large change in the distribution over labels.
We use the largest eigenvalue of the FIM as a score to quantify the “difﬁculty” of an example.
We now propose the following algorithm to compute the FIM:
After getting the eigenvalues of the FIM, we can use the largest eigenvalue λ max to quantify how fragile an example is to linguistic perturbation.
At points with largest eigenvalues, smaller perturbations can be much more effective in changing the classiﬁer output.
These examples, thus, are also more confusing and more difﬁcult for the model to classify. [Table] Input: x, f Output: λ max
Calculate probability vector : 1: p = f ( x )
Calculate Jacobian of log probability w.r.t x 2: J = ∇ x logp
Duplicate probability vector along rows to match J’s shape 3: p = duplicate Compute the FIM 4: =
Perform eigendecomposition to get the eigenvalues
6: return 3. G RADIENT A TTRIBUTION BASED P ERTURBATION In our perturbations, we rely on Integrated Gradients (IG) by Sundararajan et al.
(2017), since IG satisﬁes both sensitivity (network gradients to focus on relevant input feature attributes of the model with respect to the output) and implementation invariance of the neural network.
IG allows us to assign relative importance to each word token in the input.
After computing the token attributions with IG, we only perturb those words for “easy” and “difﬁcult” examples.
Once we compute the feature attribution for each input word in the sentence, we ﬁnd the most important words by thresholding the attribution score.
We then test the classiﬁer’s fragility by substituting these words for their synonyms/antonyms, etc, and checking if the classiﬁer prediction changes.
We show that easy examples are robust to signiﬁcant edits with multiple positive tokens like “excellent”, “great” substituted simultaneously, and “difﬁcult” examples change predictions from positive to negative with meaningless substitutions like names of actors and actresses.
D ISCUSSION AND R ESULTS 4. FIM REFLECTS DISTANCES FROM THE DECISION BOUNDARY
We ﬁrst investigate the FIM properties by training a neural network on a synthetic mixture of gaussians dataset.
The parameters of the two gaussians are µ 1 =
[ − 2 , − 2] and µ 2 =
The covariances are Σ 1 = eye (2) and Σ 2 =
The dataset is shown in ﬁgure 1.
We train a 2-layered network to separate the two classes from each other.
We use algorithm 1 to compute the largest eigenvalue of the FIM for each datapoint and use it to color the points.
We also plot the eigenvector for the top 20 points.
As seen by the gradient of the colors in Figure 1, the points with the largest eigenvalue of the FIM lie close to the decision boundary.
These points are indicative of how confusing the example is to the neural network since a small shift along the eigenvector can cause a signiﬁcant change in the KL divergence between the probability distribution of the original and new data points.
These points with a high eigenvalue are close to the decision boundary, and these examples are most susceptible to perturbations.
4. FIM VALUES CAPTURE RESILIENCE TO LINGUISTIC PERTURBATIONS , Q UALITATIVE E XPLORATION 4.2. CNN
For the difﬁcult examples, we tried the following trivial substitutions one at a time: a) a synonym b) a semantically equivalent word c) an antonym d) substituting the name of an actress present in the same passage.
As seen in Table 1, either replacing favorite with preferred or “Kirsten Dunst” with any of the listed actors/actresses sufﬁces to change the classiﬁer’s prediction.
Note that, “Megan Fox’s” name appears in the same review in the previous sentence.
Similar in Table 3, for counterfactual examples, it’s sufﬁcient to replace “exciting” with either an antonym (“boring” or “uninteresting”) or a synonym (“extraordinary” or “exceptional”).
We see the same pattern in contrast sets in Appendix.
For easy examples however, despite trying to replace four or more high attribution words simultaneously with antonyms, the predicted sentiment did not change.
Substitutions include “good” to “bad”, “unfunny” to “funny”, “factually correct” to “factually incorrect”.
Even though the passage included words like “boredom”, a word that is generally associated with a negative movie review, the model did not assign it a high attribution score.
Consequently, we did not try to substitute these words for testing robustness or fragility of word substitutions.
4.2. BERT
BERT: Transfer learned models like BERT capture rich semantic structure of the sentence.
They are robust to changes like actor names and tend to rely on semantically relevant words for classifying movie reviews.
As we can see from Table 6 difﬁcult BERT examples, even with multiple positive words, tend to predict a negative sentiment when only one of the positive word is substituted.
Even with words like “fantastic”, “terriﬁc” and “exhilarating”, just changing “best” to “worst” changed the entire sentiment of the movie review.
Easy examples for BERT require multiple simultaneous word substitutions to change the sentiment as can be seen in Table 6.
Unlike CNN models, BERT is signiﬁcantly more robust to meaningless substitutions like actor names. 4. H IGH FIM EXAMPLES ARE CHALLENGING TO NLP CLASSIFIERS , Q UANTITATIVE E XPERIMENTS 4.3. B AG OF T RICKS FOR EFFICIENT TEXT CLASSIFICATION When the dataset size is increased, simple models perform competitively to models like BERT Wang et al.
We use the bag of tricks for efﬁcient text classiﬁcation Joulin et al.
(2017) for its inference and training speed and strong performance on text classiﬁcation on 5 popular text classiﬁcation datasets (YelpReviewPolarity, AGNEWS, SogouNews, YelpReviewFull and Yahoo
Answers) In order to investigate the relationship between classiﬁer performance and FIM eigenvalues we select n-smallest and n-largest ﬁm examples from the test set. We randomly chose a perturbation strength between 0 and 1 and we perturb the example along the largest eigenvector proportional to the chosen perturbation strength. As can be seen from ﬁgure 2 left, despite sampling a large range of difﬁcult examples, between (125-1200), the classiﬁer performed poorly on the difﬁcult set with the accuracy failing to exceed 9%.
On the easy set however, the classiﬁer accuracy never falls below
In ﬁgure 2b, we plot the classiﬁer accuracy as a function of perturbation strength (l2 norm of perturbation vector for difﬁcult examples, even small perturbations upto 0.4 l2 norm, cause a 17% reduction in accuracy.
For easy examples however even upto a l2 norm of 1 barely result in any drop in performance.
More examples are in Appendix A. We randomly sample 500 examples and perturb the examples along the largest eigenvector. We use binary search to determine the minimum perturbation strength sufﬁcient to ﬂip the classiﬁer output. As we can see from Figure 3, a correlation of -0.4 implies it is possible to misclassify larger FIM examples by applying a small perturbation strength.
In order to investigate the relationship between
FIM and random word substitutions, we ﬂipped 10% of the words in a document, for 500 randomly sampled examples.
As seen in Figure 3b, a correlation of 0.35 implies that difﬁcult examples, (larger
FIM eigenvalues) are more susceptible to random word substitutions. 4. D O CONTRAST SETS AND COUNTERFACTUAL EXAMPLES LIE CLOSE TO THE DECISION BOUNDARY ? We ﬁrst plot the distribution of eigenvalues of IMDb examples, counterfactual examples and contrast sets examples in Figure 2. If the goal is to evaluate examples near the decision boundary the contrast set/counterfactual eigenvalue distribution should be shifted to the right with very little overlap with the original examples.
However, as we can see from Figure 2, we see a 69.38% overlap between contrast sets and IMDb examples as well as a 73.58% overlap between counterfactual and IMDb examples for the CNN model that was only trained on IMDb training set.
The presence of signiﬁcant overlap between the three distributions indicate that counterfactual/contrast examples are not more difﬁcult for the model compared to the normal examples.
Furthermore, since the FIM capture distance from the decision boundary, most counterfactual and contrast sets lie as far away from the decision boundary as the original IMDb dataset examples.
We next quantify the effect of changing the original sentence to the perturbed sentence in counterfactual/contrast test sets.
We plot the distribution of difference in eigenvalue of the original and perturbed sentence in Figure 2.
30% of the counterfactual examples (area to the right of 0) did not increase the FIM eigenvalue and the perturbation was ineffective.
For contrast sets, this number is around 70% (area to the right of 0), hinting that majority of perturbations are not useful for testing model robustness.
A subtle point to note here, even though 70% of counterfactual examples increase the difﬁculty, it’s because we have chosen a weak threshold (0) to quantify the usefulness of the perturbation.
A more practical threshold like 1 would lead to lesser number of useful examples for both counterfactual and contrast set examples.  I MPLICATIONS 5. C ONSTRUCTING T EST SET BY PERTURBING HIGH FIM EXAMPLES
As evident from ﬁgure 2, high FIM examples are particularly challenging for NLP classiﬁers.
This makes them useful to understand model performance.
By sampling the high FIM examples and perturbing them slightly in embedding space, we can construct a new test set to evaluate NLP models.
By repeating this process multiple times, more robust classiﬁers can be created.
5. NLP MODELS SHOULD BE EVALUATED AT THE EXAMPLES NEAR THE DECISION BOUNDARY
Deep learning models because of their high representation capacity are good at memorizing the training set.
In the absence of sufﬁcient variation in the test set, this can lead to an inﬂation in accuracy without actual generalization.
However, examples near the decision boundary of a classiﬁer, show the fragilities of these classiﬁers.
These are the examples that are most susceptible to shallow feature learning and thus are the ones that need to be tested for fragility and word substitutions.
Our approach based on FIM score, provides a task and architecture agnostic approach to discovering such examples.
[Table]
5. I NVESTIGATE HIGH FIM EXAMPLES WITH I NTEGRATED G RADIENTS TO UNDERSTAND MODEL FRAGILITY
For our models, difﬁcult examples have a mix of positive and negative words in a movie review.
The models also struggled with examples of movies that selectively praise some attributes like acting (e.g., "Exceptional performance of the actors got me hooked to the movie from the beginning") while simultaneously use negative phrases (e.g., "however the editing was horrible").
Difﬁcult examples also have high token attributions associated with irrelevant words like "nuclear," "get," and "an."
Thus substituting one or two words in difﬁcult examples change the predicted label of the classiﬁer.
Similarly, easier examples have clearly positive reviews (e.g., "Excellent direction, clever plot and gripping story").
Combining integrated gradients with high FIM examples can thus yield insights into the fragility of NLP models. 5. P ERTURBATION SHOULD BE PERFORMED ALONG THE LARGEST EIGENVECTOR
The eigenvector corresponding to λ max represents the direction of largest change in probability distribution over output.
This makes it a useful candidate direction to evaluate model fragility while random word substitutions are easier to interpret, they might end up being orthogonal to the largest eigenvector and thus not capture model fragility effectively.
One strategy to circumvent this issue is to ﬁnd word substitutions with a large projection on this eigenvector.
C ONCLUSION
We have proposed a geometrical method to quantify the difﬁculty of an example in NLP.
Our method identiﬁed fragilities in state of the art NLP models like BERT and CNN.
By directly modeling the impact of perturbation in natural language through the Fisher information metric, we showed that examples close to the decision boundary are sensitive to meaningless changes.
We also showed that counterfactual examples and contrast sets don’t necessarily lie close to the decision boundary.
Furthermore, depending on the distance from the decision boundary, small innocuous tweaks to a sentence might actually correspond to a large change in the embedding space.
Thus one has to be careful in constructing perturbations, and more disciplined approaches are needed to address the same.
As our methods are agnostic to the choice of architecture or dataset, in future we plan to extend this to other NLP tasks and datasets.
We are also studying the properties of the decision boundary: whether different classiﬁers have a similar decision boundary, are there universal difﬁcult examples which confound multiple classiﬁers?
We also plan to investigate strategies for automatic generation of sentence perturbations based on the largest eigenvector of the FIM.
Even though we have explored difﬁcult examples from a classiﬁer’s perspective, we are also interested in exploring the connections between FIM and perceived difﬁculty of examples by humans. A PPENDIX A CNN and BERT: We train a convolutional neural network (CNN) with a 50d GloVe embedding on the IMDb dataset, and calculate the eigenvalue of the FIM for each example. The accuracy on the IMDb test set was around 85.4%.
For all experiments in the paper the model was trained on the originalexamples in the original IMDb training split with a 90% train and 10% valid split.
For all experiments in the paper, we did not train on the counterfactual or contrast set examples to fairly evaluate the robustness to perturbations of contrast sets and counterfactual examples.
For BERT we ﬁnetuned ’bert-base-uncased’ and achieved an accuracy of 92.6% using huggingface transformers by Wolf et al.
We evaluated the largest eigenvalue of the dev and test sets of the contrast set and counterfactual examples datasets of IMDb.
Bag of Tricks for efﬁcient text classiﬁcation:
We use unigram and bigram word embeddings.
The latent dimension is 32.
We use torchtext datasets directly available for the training and test splits.
We use the SGD optimizer with a learning rate ofand train each model for 8 epochs.
We use a learning rate scheduler with a gamma of 0.9.
A PPENDIX B
So check if the prediction ﬂipped.
be that the classiﬁer accuracy remains very for easy examples and is signiﬁcantly low for difﬁcult examples.
the easy examples the minimal performance the weight of the perturbation along the is increased
As we can see there is a linear relationship between log lambda max and the perturbation strength needed to ﬂip the classiﬁer output.
We use binary search between the range (0,6) to discover the minimal l2 norm perturbation along the largest eigenvector which can successfully ﬂip the classiﬁer’s output.
For each dataset, we select 500 random examples from the test set for the perturbation.
Note the negative slope in all the datasets.
The p-value and the r-value are reported at the top.
The log of the largest ﬁm eigenvalue is on the x-axis.
Thus, an example with a small eigenvalue requires a larger perturbation to ﬂip classiﬁer prediction than an example with large ﬁm eigenvalue.
[Figure]
[Table]
Probably the best Dolph ﬁlm ever.
There’s everything you’d want or expect here.
Spend your time.
Dolph portrays a miserable cop with lots of interests in life.
His brother gets killed and
Dolph attempts to ﬁgure things out.
The character is just plain amazing .
[Table]
quality is really poor.
I want to write ’the jokes didn’t appeal to me’.
but the reality is for them to appeal to you, you’d have to be the man who wrote them.
or a retard.
So imagine that in script form...and this guy got THAT green lit.
Thats impressive isn’t it?
[Table]
It really impresses me that it got made.
The director/writer/actor must be really charismatic in reality.
I can think of no other way itd pass script stage.
What I want you to consider is this...while watching the ﬁlms I was feeling sorry for the actors.
It felt like being in a stand up comedy club where the guy is dying on his feet and your sitting there, not enjoying it, just feeling really bad for him coz
hes of trying.
Id really like to know what the budget is, guess it must have been low as the ﬁlm quality is really poor.
I want to write ’the jokes didn’t appeal to me’.
but the reality is for them to appeal to you, you’d have to be the man who wrote them.
or a retard.
So imagine that in script form...and this guy got THAT green lit.
Thats impressive → weird isn’t it?
[Table]
[Table]
row is the counterfactual sentence.
The counterfactual sentence is easier than the original sentence.
Note the original example relied more on words like “unlucky”, “doesn’t”, and “absolutely” during classiﬁcation.
“unlucky” and “doesn’t” are associated with more negative sentences and thus the counterfactual example is much easier for the model along with very negative words like “terrible” and “boring”.
[Table]
An excellent movie about two cops loving the same woman.
One of the cop (Périer) killed her, but all the evidences seems to incriminate the other (Montand).
The unlucky Montand doesnt know who is the other lover that could have killed her, and Périer doesnt know either that Montand had an affair with the girl.
Montand must absolutely ﬁnd the killer...and what a great ending!
Highly recommended.
[Table]
A terrible movie about two cops loving the same woman.
One of the cop (Périer) killed her, but all the evidences seems to incriminate the other (Montand).
The unlucky Montand doesnt know who is the other lover that could have killed her, and Périer doesnt know either that Montand had an affair with the girl.
Montand must absolutely ﬁnd the killer...and what a boring ending!
I don’t recommend at all.
[Table]
Was ﬂipping around the TV and HBO was showing a double whammy of unbelievably horren- dous medical conditions, so I turned to my twin sister and said, "Hey this looks like fun," - truly
I love documentaries - so we started watching it.
At ﬁrst I thought Jonni Kennedy was a young man, but then it was explained that due to his condition, he never went through puberty, thus the high voice and smaller body.
He was on a crusade to raise money for his cause.
He had the most wonderful sense of humor combined with a beautiful sense of spirituality...
I cried, watched some more, laughed, got up to get another Kleenex, then cried some more.
Once Jonni Kennedy’s "time was up" he ﬂew to heaven to be with the angels.
He was more than ready; he had learned his lessons from this life and he was free.
I highly recommend this.
If you do not fall in love with this guy, you have no heart.
[Table]
Was ﬂipping around the TV and HBO was showing a double whammy of unbelievably horren- dous medical conditions, so I turned to my twin sister and said, "Hey this looks like fun," - truly
I love documentaries - so we started watching it.
At ﬁrst I thought Jonni Kennedy was a young man, but then it was explained that due to his condition, he never went through puberty, thus the high voice and smaller body.
He was on a crusade to raise money for his cause.
He had the worst sense of humor combined with an ugly sense of spirituality...
I nodded off, watched some more, snoozed, got up to get a coffee, then snoozed some more.
Once Jonni Kennedy’s "time was up" he ﬂew to heaven to be with the angels.
He was more than ready; he had learned his lessons from this life and he was free.
I highly recommend you don’t watch this.
If you do not fall asleep within the ﬁrst ten minutes, you have no taste.
[Table]
Here’s another ﬁlm that doesn’t really need much of a recommendation.
It’s a classic comedy, very funny and entertaining and which, of course, ultimately inspired a successful television series which many would say was even better (I enjoy both, personally).
<br /><br
/>For some, it’s hard to warm up to Jack Lemmon and Walter Matthau as Felix Unger and Oscar Madison when they were were weaned on the TV show starring Tony Randall and Jack Klugman (or perhaps vice versa).
But what we’ve got there in both cases are four good actors who in real life seemed so much like their ﬁlm counterparts that they managed to make these characterizations their own.
It’s Neil Simon’s humorous material that’s key, and where the laughs really originate from.<br /><br
/>For those who have somehow never heard of THE ODD COUPLE, it’s the story of a neurotic and fussy neat-freak (Lemmon) who is thrown out of a 12-year marriage by his long-suffering wife and takes up residence in the Manhattan apartment of his sloppy and totally irresponsible buddy (Matthau).
Pitting these two unlikely roommates together within the same four walls makes for some hugely funny predicaments.
[Table]
Here’s another ﬁlm that really needs a recommendation to watch.
It’s a travesty, unfunny and which, of course, ultimately inspired a unsuccessful television series which many would say was even worse (I hated both, personally).
<br /><br
/>For some, it’s hard to warm up to Jack Lemmon and Walter Matthau as Felix Unger and Oscar Madison when they were were weaned on the TV show starring Tony Randall and Jack Klugman (or perhaps vice versa).
I am no exception.
What we’ve got there in both cases are four bad actors who in real life seemed so much unlike their ﬁlm counterparts that they managed to make these characterizations their own.
It’s Neil Simon’s material that’s the worst, and where the fails really originate from.<br /><br
/>For those who have somehow never heard of THE ODD COUPLE, it’s the story of a neurotic and fussy neat-freak (Lemmon) who is thrown out of a 12-year marriage by his long-suffering wife and takes up residence in the Manhattan apartment of his sloppy and totally irresponsible buddy (Matthau).
Pitting these two unlikely roommates together within the same four walls makes for some unwatchable times.
[Table]
This is a pathetic → excellent political satire.
No wonder why it was largely ignored in the U
[Table]
S.: it ridicules our foreign policy and misrepresents what it really is.<br /><br
/>Another bad → good ﬁlm from this era, Rendition, was however totally dismissed simply because it showed, accurately, that the U
[Table]
S. is a war machine bent on torturing, murdering, and maiming civilians in its quest for total world domination.<br /><br />A
factually incorrect → correct , bad → good acting, some big stars (John Cusack, Ben Kingsley, Marisa Tomei anyone?) and some scenes of hilarity but they couldn’t have made this movie a hit.
Thankfully, Americans don’t like to hear misrepresentations about anyone, even if they are complicit in mass murder.
[Table]
I’m a big Porsche fan, and the car was the best star in this ﬁlm.<br /><br />Haim, the drug abusing child star of the 80’s is amazing → horrible, excellent as per usual, and commenting on back up from minor characters/actors would be pointless; needless to say they were all above average.
It’s a cool movie as a trip down memory lane into the 80’s - with some weird clothes, some good shots of the Colorado backdrop and a very mind stimulating plot.<br /><br />All
in all, please watch this unless you hate 80’s movies, Corey Haim, or unlike myself, hate old school Porsches (this one in particular looks great) because life’s too short to instead watch crappy movies.
[Table]
The ﬁrst ﬁlm was an okay one, and it is nowhere as good as the wonderful animated classic which I found more poignant and endearing.
This sequel is not just inferior, its really bad.
Yes the slapstick is too much, the script has its weak spots and the plot is a tad uninspired.
Yeah probably the dogs are very cute here, but Eric Idle is dumb as a cow.
The ﬁlm is a pain to look at with any cinematography and eye hurting costumes(especially Cruella’s), and the music is sleepy.
The acting is mostly very bad, Ioan Gruffudd is appalling → terrible, shocking, excellent, extraordinary and Gerard Depardieu while he has given better performances has little fun as Cruella’s accomplice.
But the best asset, as it was with the ﬁrst ﬁlm, is the amazing Glenn Close in a deliciously over-the-top performance as Cruella, even more evil than she was previously.
Overall, poor. 3/10 Bethany Cox
[Table]
Many people here say that this show is for kids only.
Hm
[Table]
when I was a kid (approximately 7-9 years old) I watched this show ﬁrst.
It was disgusting → okay for me.
I talked with other kids about this and, sure, other shows and know what?
This was the measure of disguise, whenever we wanted to emphasize something’s silliness (either on TV or anything else)
we said ’Uh, just like Power Rangers’ and laughed.
<br /><br />And
before visiting this site I could not imagine that there actually are fans of MMPR.
It was so strange for me that I decided to watch it again and try to understand why people like it.
I did not enjoy that viewing.
But it dawned upon me: maybe I have not enough imagination?
It may be.
However this argument is not sufﬁcient for me to rate it more thanstar
[Table]