## 6 Evaluation
### Evaluation Setup
#### 6.1.1 Scoring and Ranking  
Now that we have collected data and computed various metrics, we aggregate attributes to determine the desirability of a specific service. Figure 1: INTC (Intel Corp.) transaction volume change with respect to previous trading day. Figure 2: INTC histogram of trading volume daily change, starting from 13 October 2009 over 60 trading days. The dashed vertical line is the 10% threshold. ART(\(\Delta\)V60,10%)=19. The scoring function computes a score for each stock: for each day in the snapshot aggregation approach and for a range of days in the Dynamis aggregation. The scoring function employs a weight vector (user preferences) in the snapshot aggregation approach and a weight vector, as well as a threshold vector in the Dynamis aggregation approach. Let \(\vec{w}=(w_{1},\ldots,w_{n})\) be the weight vector and \(\vec{\tau}=(\tau_{1},\ldots,\tau_{n})\) be the threshold vector. Let \(s=\langle a_{1},\ldots,a_{n}\rangle\) be some service. For snapshot aggregation, the scoring function (SAgg) is:  
\[SAgg(s)=\sum_{i=1}^{n}w_{i}\cdot a_{i}\]  
For Dynamis aggregation, the scoring function (DAgg) is:  
\[DAgg(s)=\sum_{i=1}^{n}w_{i}\cdot ART(a_{i},\tau_{i})\]  
In both the snapshot and Dynamis aggregation, we use a linear combination of the attributes available to score the services. For our stock setting, we picked:  
\[\vec{w}=(2.0,0.1,3.0)\]  
These three weights correspond to \(\Delta\)P, \(\Delta\)V and RSD. The values reflect our opinion that the relative performance of a stock against a benchmark is slightly more important than the daily percentage increase, and the volume percentage change is least important. Figure 3: Evolution of the ART(\(\Delta\)V60,10%) integer-valued function over time for INTC. The score computed with (_SAgg_) for INTC as it varies over time is depicted in Figure 4. For the Dynamis aggregation approach, the resulting scores vary more slowly over time (_cf._, Figure 5). In our dataset, we have scored every stock on every day. So far, we have looked at the score of a single stock as it changes over time. We can examine the scores for all of the stocks on a specific day, sorted  
Figure 4: INTC score computed daily using snapshotting with the scoring function: \(SAgg(\text{INTC})=3*\text{RSD}+2*\text{AP}+0.1*\text{AV}\). Figure 5: INTC score computed daily using Dynamis with the following scoring function: \(\text{DAgg}(\text{INTC})=3*\text{ART}(\text{RSD}60,1\%)+2*\text{ART}(\Delta \text{P}60,2\%)+0.1*\text{ART}(\Delta\text{V}60,10\%)\). from best to worst. The position of a stock in this sorted list is its rank for that day; therefore, a high score results in a low rank: rank 1 is the best stock pick. On a specific date, say 3 October 2011, the curve depicted in Figure 6 represents the sorted stocks, and Table 2 lists the top 10 ranked stock tickers for that day. These present the ranking of tickers on a specific date, but the evolution of a given ticker's rank over time can also be graphed. Figure 1 and Figures 3-5 are time-based representations of the data. In contrast, Figure 6 represents the stock tickers on a fixed date. The following figures are again plotted against time. In the case of INTC, the results, which are based on the snapshot and Dynamis aggregation techniques, are shown in \begin{table}
\begin{tabular}{r c l} \hline \hline
**Rank** & **Ticker** & **Company** \\ \hline
1 & CF & CF Industries Holdings, Inc. \\
2 & RRC & Range Resources Corp. \\
3 & RHT & Red Hat, Inc. \\
4 & TLAB & Tellabs, Inc. \\
5 & NVDA & NVIDIA Corporation \\
6 & GT & The Goodyear Tire \& Rubber Company \\
7 & JWN & Nordstrom, Inc. \\
8 & SNDK & SanDisk Corporation \\
9 & CTX & Citrix Systems, Inc. \\
10 & PFG & Principal Financial Group Inc. \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Top 10 stocks on 3 October 2011, ranked with Dynamis aggregation. These correspond to the highest-scored tickers in Figure 6.  
Figure 6: Sorted scores computed with Dynamis aggregation on 3 October 2011. Figures 7 and 8, respectively.