How does ResNet bottleneck architecture's input size is possible to change from 56x56x64 to 56x56x356? In ResNet papaer, First residual block's input size is 56x56x64 caused by 7x7x64 filter in first layer. But, in the paper, they showed residual block that has 56x56x256 input size. How does it is possible to change from 56x56x64 to 56x56x256?

I do not know the answer for sure but I assume that the "256-d" refers to the shortcut connection and not the input. Since the output dimension of the 3 conv. layers in your right hand picture (fig. 5) is 256 the shortcut is transformed from depth 64 to depth 256.

How this is being done is explained in the following paragraph on p. 4:

  Residual Network. Based on the above plain network, we
  insert shortcut connections (Fig. 3, right) which turn the
  network into its counterpart residual version. The identity
  shortcuts (Eqn.(1)) can be directly used when the input and
  output are of the same dimensions (solid line shortcuts in
  Fig. 3). When the dimensions increase (dotted line shortcuts
  in Fig. 3), we consider two options: (A) The shortcut still
  performs identity mapping, with extra zero entries padded
  for increasing dimensions. This option introduces no extra
  parameter; (B) The projection shortcut in Eqn.(2) is used to
  match dimensions (done by 1×1 convolutions). For both
  options, when the shortcuts go across feature maps of two
  sizes, they are performed with a stride of 2.

For ResNet-34 this not required for the first building block which is shown in fig. 5 but for later blocks too. This is shown by the dotted lines in figure 3:


What is the level of measurement / name of the scale of list-features? If you look at publications, you can have a dataset

title of publication
list of authors
number of pages
year of publication

The Level of measurement of "number of pages" is  interval scale, the year of publication is interval scale as well, the title is nominal. But what is the list of authors? Simply saying that it is nominal seems not to capture a major part of this attribute.

With a list such as this list of authors, the only representation which truly capture the semantics is a (long) list of boolean features, one for each possible author in the set. Ideally one would also pre-process the whole set of authors in order to match the different ways a name can be written, e.g. "J. Doe" = "John Doe" = "Doe, John".


K-Means clustering for mixed numeric and categorical data My data set contains a number of numeric attributes and one categorical.
Say, NumericAttr1, NumericAttr2, ..., NumericAttrN, CategoricalAttr,
where CategoricalAttr takes one of three possible values: CategoricalAttrValue1, CategoricalAttrValue2 or CategoricalAttrValue3.
I'm using default k-means clustering algorithm implementation for Octave.
It works with numeric data only.
So my question: is it correct to split the categorical attribute CategoricalAttr into three numeric (binary) variables, like IsCategoricalAttrValue1, IsCategoricalAttrValue2, IsCategoricalAttrValue3 ?

The standard k-means algorithm isn't directly applicable to categorical data, for various reasons.  The sample space for categorical data is discrete, and doesn't have a natural origin.  A Euclidean distance function on such a space isn't really meaningful.  As someone put it, "The fact a snake possesses neither wheels nor legs allows us to say nothing about the relative value of wheels and legs." (from here)

There's a variation of k-means known as k-modes, introduced in this paper by Zhexue Huang, which is suitable for categorical data.   Note that the solutions you get are sensitive to initial conditions, as discussed here (PDF), for instance.

Huang's paper (linked above) also has a section on "k-prototypes" which applies to data with a mix of categorical and numeric features.  It uses a distance measure which mixes the Hamming distance for categorical features and the Euclidean distance for numeric features.

A Google search for "k-means mix of categorical data" turns up quite a few more recent papers on various algorithms for k-means-like clustering with a mix of categorical and numeric data.  (I haven't yet read them, so I can't comment on their merits.)

Actually, what you suggest (converting categorical attributes to binary values, and then doing k-means as if these were numeric values) is another approach that has been tried before (predating k-modes).  (See Ralambondrainy, H. 1995. A conceptual version of the k-means algorithm. Pattern Recognition Letters, 16:1147–1157.)  But I believe the k-modes approach is preferred for the reasons I indicated above.

Mixture models can be used to cluster  a data set composed of continuous and categorical variables.
You can use the R package VarSelLCM (available on CRAN) which models, within each cluster, the continuous variables by Gaussian distributions and the ordinal/binary variables. Take care to store your data in a data.frame where continuous variables are "numeric" and categorical variables are "factor".
A tutorial is available here
Moreover, missing values can be managed by the model at hand.


What is the best way to replace NaN values for ranked columns I have a column named BsmntQual that gives a ranking on the height of the basement per each house.  These are all of the unique values in this column:

print(train['BsmtQual'].unique().tolist())

>>> ['Gd', 'TA', 'Ex', nan, 'Fa']

This is the legend of this particular column:

BsmtQual: Evaluates the height of the basement

       Ex   Excellent (100+ inches)
       Gd   Good (90-99 inches)
       TA   Typical (80-89 inches)
       Fa   Fair (70-79 inches)
       Po   Poor (<70 inches
       NA   No Basement < Not to be confused with the nan value above

This is what I did for my other ranked columns but this one did not have NaN values:

train['ExterQual'] = train['ExterQual'].replace(['Ex', 'Gd', 'TA', 'Fa'], [4, 3, 2, 1])  # Exterior Quality

For numerical values, a common way is to fill all NaN values with the mean of the column.  But what is a good way of replacing the NaN values for columns such as these?

Here is the full dataset

Your legend clearly states that missing values mean that there is no basement. You could fill the missing values with ’NoBase’ to make that point clearer (train[‘BsmtQual’].fillna(‘NoBase’, inplace=True)).
When you rank them then, you just add another ranking for ‘NoBase’, maybe 0, based on your example rating that gives a higher value to more quality.

method : {‘backfill’, ‘bfill’, ‘pad’, ‘ffill’, None}, default None
Method to use for filling holes in reindexed Series pad / ffill: propagate last valid observation forward to next valid backfill / bfill: use NEXT valid observation to fill gap

from pandas fillna, these methods can be used or fill in with the highest frequency category.


Export Python Code from Orange Since Orange is just running python code under the hood... is it possible to set up and do my analysis in Orange but then export or output the underlying code for documentation or replication?

Since I was also looking for it just yesterday being a newbie,sharing what I found , As per Orange FAQ it says

and it directs to the link @K3---rnc mentioned.

But i found similar request as well not sure how well it works-

Orange Data Mining load saved models

https://stackoverflow.com/questions/45598627/orange-save-model-in-python-script

A common complaint about Python’s syntax is that it’s impossible to generate Python code on the fly, from a program.
If you want, use Rcmdr (R pkg) to do your analysis and machine learning. You can export the output AND THE CODE.
I use since long time the two languages. Python is powerful but it has this black point.


Why using L1 regularization over L2? Conducting a linear regression model using a loss function, why should I use $L_1$ instead of $L_2$ regularization?

Is it better at preventing overfitting? Is it deterministic (so always a unique solution)? Is it better at feature selection (because producing sparse models)? Does it disperse the weights among the features?

Basically, we add a regularization term in order to prevent the coefficients to fit so perfectly to overfit.

The difference between L1 and L2 is L1 is the sum of weights and L2 is just the sum of the square of weights.

L1 cannot be used in gradient-based approaches since it is not-differentiable unlike L2

L1  helps perform feature selection in sparse feature spaces.Feature selection is to know which features are helpful and which are redundant.

The difference between their properties can be summarized as :

L2 has one very important advantage to L1, and that is invariance to rotation and scale.

This is especially important in geographical / physical application.

Say your technician accidentally installed your sensor in 45 degree angel, L1 would be affected, while L2 (Euclidean distance) would remain the same.


How can I use Machine learning for inter-relationship between Features? Machine learning is used mostly for prediction and there are numerous algorithms and packages for this.

How can I use machine learning for studying inter-relationships between features? What are major packages and functions for this? Are there any packages for graphics in this area? Can artificial neural networks also be used for this purpose? If so, any particular type is specifically suited for this?

I do not want to limit to any particular language like Python or R.

I use pair plots to study the inter-relationship between features. Pair plot gives the first level information of the features. Seaborn library has pairplot function and even matplotlib has the same.

Another thing you can use heatmap which gives the co-relation between features, by using heat map we can see features that are high co-related and may eliminate one of them. As a word of caution you must good reason or domain knowledge to drop a feature

I guess Ridge regression technique is used when the data suffers from multicollinearity.It solves the multicollinearity problem using the shrinkage parameter (lambda).

Also, the presence of multicollinearity does not affect the predictive capability of the model