Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper introduces a new logmoid activation unit, that has nice properties to approximate continuous function and introduce performance gain over other activation functions across multiple datasets.
strength_and_weaknesses: Strength:
1. an interesting activation function with two parameters that covers the geometry of existed activation functions.

2. theoretical analysis in terms of approximating continuous functions.

3. excellent results achieved over multiple network structures and datasets.

Weakness:
the experiments are far from extensive.
1. we do not know if this activation works for other tasks like detection/segmentation.
2. it is unknown if this activation works for transformer.
3. it is unknown if this activation works well with dynamic operators like SE in MobileNet-V3.
4. it is unknown if this activation works well for large scale dataset (ImageNet-1K, ImageNet-22K).

clarity,_quality,_novelty_and_reproducibility: I like the idea and the theoretical analysis, but not satisfied with the experiments. A good activation function should be able to work well for multi-tasks, multi-architectures and large scale dataset, which is not validated in the paper. 
summary_of_the_review: This paper is borderline: interesting idea, good analysis, but weak experiments (see weakness)
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes a new family of activation functions called LAUs (Logmoid Activation Units). LAUs have two parameters α and β, and the authors comprehensively discuss the change of LAUs and their derivatives with different α and β. The authors also prove that Logmoid-1 (a special case of LAUs) makes Feed-forward neural network achieves lower approximation error than Swish with both mathematical and experimental results. The authors show that LAUs perform better than baseline activation functions on different datasets and architectures.
strength_and_weaknesses: Strength:
The proposed LAUs are simple and easy to implement.

Weakness:
1.	Insufficient experiments. Pade is introduced in Section Related Works, but the authors did not compare LAUs with Pade among all the benchmark experiments. Besides, there are too many experiments on tiny datasets and networks, making the improvements not very impressive and convincing. Similar to Pade, I suggest the authors provide results of LAUs on ImageNet-1K with MobileNet-V2. Furthermore, there are many powerful activation functions like ACON (Activate or Not: Learning Customized Activation) and PWLU (Learning specialized activation functions with the Piecewise Linear Unit) that the authors did not see. To make the results more convincing, I suggest authors compare LAUs with ACON and PWLU.
2.	This paper is badly written. Section 4.1 suddenly appears without any introduction, which makes readers confused. And Section 4.2 is unnecessary because Section 5 provides experiments about Logmoid-1. All the tables are not centered. Table and figure captions are not clear enough for readers to understand. 
3.	Limited novelty. There are many Sigmoid-based activation functions like Soft-Root-Sign and ACON, and it is convenient to provide different Sigmoid-based activation functions with search methods. So I do not see much novelty in this paper.

clarity,_quality,_novelty_and_reproducibility: Clarity: bad
Quality bad
Novelty: limited
Reproducibility: best

summary_of_the_review: The proposed LAUs are Sigmoid-based activation functions. The overall novelty is limited, and missed essential experiments. The paper is bad-written with the problem of duplication and lack of logical coherence.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: This paper is 16 pages (+references), while the strict upper bound for ICLR is 9 pages. Thus I treat this paper as desk rejected.
strength_and_weaknesses: Exceeds page limit.
clarity,_quality,_novelty_and_reproducibility: Exceeds page limit.
summary_of_the_review: Exceeds page limit, desk reject.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: ## Summary

This paper proposes a novel activation unit for neural networks called LAU with parametrization of $y= x \text{ln}(1+\alpha \text{sigmoid}(\beta x))$ where $\alpha$ and $\beta$ are learnable parameters that are optimized with backpropagation. This gives neural networks the ability to approximate any continuous activation function in closed spaces. The paper presents empirical evidence that the LAU outperforms ReLU on the benchmark datasets they proposed.

The authors agreed to reject this paper. The initial submission of the paper was over the page limit and it was 19 pages which was a violation of the submission guidelines. As it stands this paper is not publication-ready yet.

Below I will summarize some of the strengths and weaknesses pointed by the other reviewers.

## Strengths

- Strong experimental results.
- Simple and easy to implement.
- Theoretical results.

## Weaknesses

- Experiments are limited to classification tasks only. It is not clear if they would work for detection or segmentation only.
- The experiments are only limited to typical feedforward architectures used for classification. It is not clear if they would work with transformers or LSTMs, for instance.
- The experiments focus on small-scale datasets. It is not clear if the results would scale well to the larger datasets and larger models.
- The paper is badly written.
- Initial submission is significantly over page limit.
justification_for_why_not_higher_score: The paper is badly written and the reviewers unanimously agreed to reject this paper. The initial submission was significantly over the page limits.
justification_for_why_not_lower_score: There is no lower score.
Decision: Reject