Miracle-dz/UniFAR-FTUs huggingface dataset:
Triangle-Reward Reinforcement Learning: A Visual-Linguistic Semantic Alignment for Image Captioning

Image captioning aims to generate a sentence consisting of sequential linguistic words, to describe visual units (i.e., objects, relationships, and attributes) in a given image. Most of existing methods rely on the prevalent supervised learning with cross-entropy (XE) function to transfer visual units into a sequence of linguistic words. However, we argue that the XE objective is not sensitive to visual-linguistic alignment, which cannot discriminately penalize the semantic inconsistency and shrink the context gap. To solve these problems, we propose the Triangle-Reward Reinforcement Learning (TRRL) method. TRRL uses the scene graph (G)---objects as nodes and relationships as edges---to represent images, generated sentences, and ground truth sentences individually, and mutually align them during the training process. Specifically, TRRL formulates the image captioning into cooperative agents, where the first agent aims to extract visual scene graph (Gimg) from image (I) and the second agent translates this graph into sentence (S). To discriminately penalize the visual-linguistic inconsistency, TRRL proposes the novel triangle-reward function: 1) the generated sentence and its corresponding ground truth are decomposed into the linguistic scene graph (Gsen) and ground-truth scene graph (Ggt), respectively; 2) Gimg, Gsen, and Ggt are paired to calculate the semantic similarity scores which are proportionally assigned to reward each agent. Meanwhile, to make the training objective sensitive to context changes, we propose the node-level and triplet-level scoring methods to jointly measure the visual-linguistic graph correlations. Extensive experiments on the MSCOCO dataset demonstrate the superiority of TRRL. Additional ablation studies further validate its effectiveness.
Speculative Privacy Tracking (SPT): Leaking Information From Speculative Execution Without Compromising Privacy

Speculative execution attacks put a dangerous new twist on information leakage through microarchitectural side channels. Ordinarily, programmers can reason about leakage based on the program’s semantics, and prevent said leakage by carefully writing the program to not pass secrets to covert channel-creating “transmitter” instructions, such as branches and loads. Speculative execution breaks this defense, because a transmitter might mis-speculatively execute with a secret operand even if it can never execute with said operand in valid executions. This paper proposes a new security definition that enables hardware to provide comprehensive, low-overhead and transparent-to-software protection against these attacks. The key idea is that it is safe to speculatively execute a transmitter without any protection if its operands were already leaked by the non-speculative execution. Based on this definition we design Speculative Privacy Tracking (SPT), a hardware protection that delays execution of every transmitter until it can prove that the transmitter’s operands leak during the program’s non-speculative execution. Using a novel dynamic information flow analysis microarchitecture, SPT efficiently proves when such an operand declassification implies that other data becomes declassified, which enables other delayed transmitters to be executed safely. We evaluate SPT on SPEC2017 and constant-time code benchmarks, and find that it adds only 45%/11% overhead on average (depending on the attack model) relative to an insecure processor. Compared to a secure baseline with the same protection scope, SPT reduces overhead by an average 3.6 × /3 ×.
Get The Best of the Three Worlds: Real-Time Neural Image Compression in a Non-GPU Environment

Lossy image compression always faces a tradeoff between rate-distortion performance and compression/decompression speed. With the advent of neural image compression, hardware (GPU) becomes the new vertex in the tradeoff triangle. By resolving the high GPU dependency and improving the low speed of neural models, this paper proposes two non-GPU models that get the best of the three worlds. First, the CPU-friendly Independent Separable Down-Sampling (ISD) and Up-Sampling (ISU) modules are proposed to lighten the network while ensuring a large receptive field. Second, an asymmetric autoencoder architecture is adopted to boost the decoding speed. At last, the Inverse Quantization Residual (IQR) module is proposed to reduce the error caused by quantization. In terms of rate-distortion performance, our network surpasses the state-of-the-art real-time GPU neural compression work at medium and high bit rates. In terms of speed, our model's compression and decompression speeds surpass all other traditional compression methods except JPEG, using only CPUs. In terms of hardware, the proposed models are CPU friendly and perform stably well in a non-GPU environment. The code is publicly available at https://github.com/kengchikengchi/FasiNet.
Soteria: Towards Resilient Integrity-Protected and Encrypted Non-Volatile Memories

Although emerging Non-Volatile Memories (NVMs) are expected to be adopted in future memory and storage systems, their non-volatility brings complications in designing processors wherein security is an essential requirement. One of these complications is maintaining the correctness of the security metadata for encryption and integrity verification. Due to the accommodation of security metadata in the NVMs, they are susceptible to reliability threats posed by the underlying memory technology. This is undesirable because the secure operation of the system highly depends on the correctness of the security metadata stored in the memory. We observe that the error sensitivity of security metadata is higher than general data and requires special attention. A single uncorrectable error in a top Merkle tree node can leave a large portion of memory data unverifiable. To solve this, we propose Soteria, a scheme that provides higher error-tolerance of security metadata by lazily duplicating them. Soteria decouples existing memory reliability from the security metadata reliability and achieves security, performance, and high reliability within the same system with only minor memory controller changes. Our proposed scheme improves the reliability of the improved security NVM system significantly while causing only about 1% system slowdown on average.
Style-Aware Image Recommendation for Social Media Marketing

Social media have become a popular platform for brands to allocate marketing budget and build their relationship with customers. Posting images with a consistent concept on social media helps customers recognize, remember, and consider brands. This strategy is known as brand concept consistency in marketing literature. Consequently, brands spend immense manpower and financial resources in choosing which images to post or repost. Therefore, automatically recommending images with a consistent brand concept is a necessary task for social media marketing. In this paper, we propose a content-based recommendation system that learns the concept of brands and recommends images that are coherent with the brand. Specifically, brand representation is performed from the brand posts on social media. Existing methods rely on visual features extracted by pre-trained neural networks, which can represent objects in the image but not the style of the image. To bridge this gap, a framework using both object and style vectors as input is proposed to learn the brand representation. In addition, we show that the proposed method can not only be applied to brands but also be applied to influencers. We collected a new Instagram influencer dataset, consisting of 616 influencers and about 1 million images, which can greatly benefit future research in this area. The experimental results on two large-scale Instagram datasets show the superiority of the proposed method over state-of-the-art methods.
Syntropic Counterpoints: Metaphysics of The Machines

In the artwork Syntropic Counterpoints: Metaphysics of The Machines, we tend to explore phenomena of AI aesthetic and challenge machine abstraction. Our approach toward the liberation of machine creativity is through the use of words and grammar as a creative tool humans developed to express worlds "beyond" the world, existing and non-existing realities. We are lead by Nietzsche's claim that grammar is the "Metaphysics of the People," as such grammar, content, and vision generated during the philosophical discussion between our AI clones is "Metaphysics of Machines" through we can experience their realities and start to question our own.
Seeing is Believing? Effects of Visualization on Smart Device Privacy Perceptions

Research on smart device privacy has consistently highlighted how privacy is an important concern for users, but they fail to act on their concerns. While this discrepancy between user perceptions and actions has been consistently reported, currently there is a limited understanding of why this is the case or how the situation can be ameliorated. This paper systematically studies how visualizations in privacy assistants can improve the situation, reporting on two studies that explore the users’ privacy perceptions in smart device ecosystems. The first study shows that displaying device location and data type reduces the users’ privacy perceptions. Participants also weigh the use of media such as online news as a source to inform users about the possible inferences. The second study analyzes participants’ preferences to visualize smart device information and privacy policies using augmented reality. Through these two studies, we derive insights and guidelines on how to design effective privacy assistants and to improve user’s knowledge of risks associated with data disclosure in smart home scenarios.
Archaeal origins of gamete fusion

Sexual reproduction in Eukarya consists of genome reduction by meiosis and subsequent gamete fusion. The presence of meiotic genes in Archaea and Bacteria suggests that prokaryotic DNA repair mechanisms evolved towards meiotic recombination1,2. However, the evolutionary origin of gamete fusion is less clear because fusogenic proteins resembling those found in Eukarya have so far not been identified in prokaryotes3–5. Here, using bioinformatics, we identified archaeal genes encoding candidates of fusexins, a superfamily of fusogens mediating somatic and gamete fusion in multiple eukaryotic lineages. Crystallographic structure determination of a candidate archaeal FusexinA reveals an archetypical trimeric fusexin architecture with novel features such as a six-helix bundle and an additional globular domain. We demonstrate that ectopically expressed FusexinA can fuse mammalian cells, and that this process involves the additional domain and a more broadly conserved fusion loop. Genome content analyses reveal that archaeal fusexins genes are within integrated mobile elements. Finally, evolutionary analyses place these archaeal fusogens as the founders of the fusexin superfamily. Based on these findings, we propose a new hypothesis on the origins of eukaryotic sex where an archaeal fusexin, originally used by selfish elements for horizontal transmission, was repurposed to enable gamete fusion.
PRMT5 in T cells drives Th17 responses, mixed granulocytic inflammation and severe allergic airway inflammation

Severe asthma is characterized by steroid insensitivity and poor symptom control, and is responsible for the majority of asthma-related hospital costs. Therapeutic options remain limited, in part due to limited understanding in mechanisms driving severe asthma. Increased arginine methylation, catalyzed by protein arginine methyltransferases (PRMTs), is increased in asthmatic lungs. Here, we show that PRMT5 drives allergic airway inflammation in a mouse model reproducing multiple aspects of human severe asthma. We find that PRMT5 is required in CD4+ T cells for chronic steroid-insensitive severe lung inflammation, with selective T cell deletion of PRMT5 robustly suppressing eosinophilic and neutrophilic lung inflammation, pathology, airway remodeling and hyperresponsiveness. Mechanistically, we observed high pulmonary sterol metabolic activity, ROR-γt and Th17 responses, with PRMT5-dependent increases in ROR-γt’s agonist desmosterol. Our work demonstrates that T cell PRMT5 drives severe allergic lung inflammation and has potential implications for the pathogenesis and therapeutic targeting of severe asthma.
A Pseudorabies Virus Serine/Threonine Kinase, US3, Promotes Retrograde Transport in Axons via Akt/mToRC1

Neurons are highly polarized cells with axons that can reach centimeters in length. Communication between axons at the periphery and the distant cell body is a relatively slow process involving the active transport of chemical messengers. ABSTRACT Infection of peripheral axons by alpha herpesviruses (AHVs) is a critical stage in establishing a lifelong infection in the host. Upon entering the cytoplasm of axons, AHV nucleocapsids and associated inner-tegument proteins must engage the cellular retrograde transport machinery to promote the long-distance movement of virion components to the nucleus. The current model outlining this process is incomplete, and further investigation is required to discover all viral and cellular determinants involved as well as the temporality of the events. Using a modified trichamber system, we have discovered a novel role of the pseudorabies virus (PRV) serine/threonine kinase US3 in promoting efficient retrograde transport of nucleocapsids. We discovered that transporting nucleocapsids move at similar velocities in both the presence and absence of a functional US3 kinase; however, fewer nucleocapsids are moving when US3 is absent, and they move for shorter periods of time before stopping, suggesting that US3 is required for efficient nucleocapsid engagement with the retrograde transport machinery. This led to fewer nucleocapsids reaching the cell bodies to produce a productive infection 12 h later. Furthermore, US3 was responsible for the induction of local translation in axons as early as 1 h postinfection (hpi) through the stimulation of a phosphatidylinositol 3-kinase (PI3K)/Akt-mToRC1 pathway. These data describe a novel role for US3 in the induction of local translation in axons during AHV infection, a critical step in transport of nucleocapsids to the cell body. IMPORTANCE Neurons are highly polarized cells with axons that can reach centimeters in length. Communication between axons at the periphery and the distant cell body is a relatively slow process involving the active transport of chemical messengers. There is a need for axons to respond rapidly to extracellular stimuli. Translation of repressed mRNAs present within the axon occurs to enable rapid, localized responses independently of the cell body. AHVs have evolved a way to hijack local translation in the axons to promote their transport to the nucleus. We have determined the cellular mechanism and viral components involved in the induction of axonal translation. The US3 serine/threonine kinase of PRV activates Akt-mToRC1 signaling pathways early during infection to promote axonal translation. When US3 is not present, the number of moving nucleocapsids and their processivity are reduced, suggesting that US3 activity is required for efficient engagement of nucleocapsids with the retrograde transport machinery.
Belemnite phylogeny and decline during the mid-Cretaceous

Belemnites are common fossil coleoid cephalopods of the Mesozoic. They began to diversify in the Triassic-Early Jurassic and maintained this diversity until the early Early Cretaceous. During the mid-Cretaceous, they declined in diversity and distribution, being restricted to only the Boreal and Austral Realm since the Turonian. Here, I present the first cladistic analysis of belemnite phylogeny, spanning taxa representative of the whole diversity and stratigraphic range of the group. This analysis shows that the usually applied subdivision of all belemnites into “Belemnitina” and “Belemnopseina” is not supported. A newly identified clade, the Pseudoalveolata, is suggested here. Pseudoalveolate belemnites represent the last remaining belemnites after the Aptian. Oceanic anoxia and warming are likely the main cause of the mid-Cretaceous belemnite decline, resulting in the Aptian-Albian dominance of the warm-adapted pseudoalveolate genus Neohibolites. The rise of teleost fish diversity during the mid-Cretaceous is discussed and its relevance for belemnite evolution. Some teleosts (e.g., Enchodus) might have taken over the mesopredator niches left by belemnites during the mid-Cretaceous, being better adapted to warming seas. Belemnites were not able to recover their earlier widespread distribution and diversity and the last remaining, disjunctly distributed families, the northern Belemnitellidae and southern Dimitobelidae, became extinct at the K/Pg-boundary.
Distinct dendritic Ca2+ spike forms produce opposing input-output transformations in rat CA3 pyramidal cells

Proper integration of different inputs targeting the dendritic tree of CA3 pyramidal cells (CA3PCs) is critical for associative learning and recall. Dendritic Ca2+ spikes have been proposed to perform associative computations in other PC types, by detecting conjunctive activation of different afferent input pathways, initiating afterdepolarization (ADP) and triggering burst firing. Implementation of such operations fundamentally depends on the actual biophysical properties of dendritic Ca2+ spikes; yet little is known about these properties in dendrites of CA3PCs. Using dendritic patch-clamp recordings and two-photon Ca2+ imaging in acute slices from male rats we report that, unlike CA1PCs, distal apical trunk dendrites of CA3PCs exhibit distinct forms of dendritic Ca2+ spikes. Besides ADP-type global Ca2+ spikes, a majority of dendrites expresses a novel, fast Ca2+ spike type that is initiated locally without backpropagating action potentials, can recruit additional Na+ currents, and is compartmentalized to the activated dendritic subtree. Occurrence of the different Ca2+ spike types correlates with dendritic structure, indicating morpho-functional heterogeneity among CA3PCs. Importantly, ADPs and dendritically initiated spikes produce opposing somatic output: bursts versus strictly single action potentials, respectively. The uncovered variability of dendritic Ca2+ spikes may underlie heterogeneous input-output transformation and bursting properties of CA3PCs, and might specifically contribute to key associative and non-associative computations performed by the CA3 network.
Effects of forest disturbance and regeneration on tree species composition and traits in DR Congo

Despite their key role in biodiversity conservation, forests of the Congo Basin are increasingly threatened by human activities. Whereas deforestation is very conspicuous, it remains challenging to monitor the impact of forest degradation under a more or less intact canopy. Likewise, the outcomes of forest regeneration following agricultural abandonment remain poorly studied in the Congo Basin. Here, we surveyed 125 vegetation quadrats across 25 forest inventory plots in the Yangambi region, and based on historical land-use maps. We aimed to assess both the impact of anthropogenic disturbance and forest recovery on tree species diversity, tree functional traits, community composition, canopy cover, and forest structure, as compared to undisturbed old-growth forest. We found that undisturbed old-growth forest harboured more tree individuals, particularly individuals with a smaller diameter, as well as species with a higher wood density when compared to both disturbed old-growth and regrowth forest. Moreover, species diversity was highest in disturbed old-growth forest. Tree species composition was significantly different between undisturbed old-growth forest and the other two forest categories. Our findings indicate that anthropogenic disturbance results in a degraded forest structure and may reduce the forest’s ability to store carbon. Whereas species diversity recovered 60 years after agricultural abandonment, species composition remained different from undisturbed old-growth forest. Our study illustrates the need for more extensive vegetation surveys in the Congo Basin in order to quantify the effects of human disturbance under a closed canopy.
APOLLO: An Automated Power Modeling Framework for Runtime Power Introspection in High-Volume Commercial Microprocessors

Accurate power modeling is crucial for energy-efficient CPU design and runtime management. An ideal power modeling framework needs to be accurate yet fast, achieve high temporal resolution (ideally cycle-accurate) yet with low runtime computational overheads, and easily extensible to diverse designs through automation. Simultaneously satisfying such conflicting objectives is challenging and largely unattained despite significant prior research. In this paper, we propose APOLLO, an automated per-cycle power modeling framework that serves as the basis for both a design-time power estimator and a low-overhead runtime on-chip power meter (OPM). APOLLO uses the minimax concave penalty (MCP)-based feature selection algorithm to automatically select less than 0.05% of RTL signals as power proxies. The power estimation achieves R2 > 0.95 on Arm Neoverse N1 [3] and R2 > 0.94 on Arm Cortex-A77 [2] microprocessors, respectively. When integrated with an emulator-assisted flow, APOLLO finishes per-cycle power estimation on millions-of-cycles benchmark in minutes for million-gate industrial CPU designs. Furthermore, the power model is synthesized and integrated into the microprocessor implementation as a runtime OPM. APOLLO’s accuracy further improves when coarse-grained temporal resolution is preferred. To our best knowledge, this is the first runtime OPM that simultaneously achieves per-cycle temporal resolution and area/power overhead without compromising accuracy, which is validated on high-performance, out-of-order industrial CPU designs.