Category Archives: Human Reproduction

(Beyond Pesticides, December 1, 2023) Even though researchers have noted since the 1970s that human fertility appears to be declining globally, doubt is still circulating that it is really happening and that pesticides could have anything to do with it. Very recently published studies, however, make it clear that, even without exact elucidation of the mechanisms by which pesticides damage male fertility, there is an unmistakable association of pesticides and many aspects of male reproductive health.

One of the new studies, a meta-analysis of 25 studies on the connection between pesticides and male reproductive problems, finds that men exposed to organophosphate (such as glyphosate and malathion) and carbamate (such as carbaryl and methiocarb) insecticides have lower sperm concentrations than the general population. This is especially true of men exposed in work settings. The senior author of the study, Melissa J. Perry,ScD of the George Mason University College of Public Health, told HealthNews, The evidence available has reached a point that we must take regulatory action to reduce insecticide exposure.

Human infertility is defined as the failure to achieve pregnancy after 12 months or more of regular unprotected sexual intercourse. Most public attention regarding infertility focuses on womens difficulties in getting pregnant, causing couples to resort to in vitro fertilization and surrogates. But about a third to half the time, a couples infertility results from problems with the male contribution. Mens reproductive health is measured by total sperm count, sperms ability to move, the incidence of malformed sperm or reproductive organ structure, testosterone levels and other criteria.

The relationships between aspects of male reproductive health such as sperm count, fertility and testicular cancer are not perfectly understood, but they are known to be interrelated. Low sperm counts can not only indicate decreased fertility, but also correlate with other markers of declining male reproductive health, including testicular tumors and testosterone levels. In 2017 Shanna Swan, PhD of the Icahn School of Medicine at Mount Sinai and colleagues published a major review of changes in sperm count between 1973 and 2011. They found that sperm counts declined by 52.4 percent over their study period.

Swan et al. also noted that reduced sperm count is a strong predictor of overall disease and death risk. In other words, sperm count reflects influences on health that go far beyond reproduction, and also that reproductive health is created by proper hormone balance, which many pesticides are well known to disturb.

Dr. Swan and colleagues wrote that chemical exposures, including pesticides (especially the endocrine disrupters) are plausible bad actors in the sperm count decline, but also said lifestyle factors such as diet and smoking are likely factors. High body mass index (BMI) and obesity have also been associated with low sperm counts.

Obesity is often cited as a lifestyle choice causing the reproductive problems, unrelated to factors like pesticide exposures. This is something of a straw man, however, because obesity itself can be an outcome of such exposures. For example, a 2022 review found that two carbamate insecticides and eight organophosphate insecticides were significantly associated with higher obesity prevalence, suggesting that obesity and low sperm count may have a common cause rather than a direct cause-and-effect relationship.

Pesticides present an especially vexing problem in that they affect organisms through many different pathways, often simultaneously. For example, organophosphates notoriously damage neurotransmitters, but they have also been associated with poor semen quality in exposed factory workers. Similarly, carbamates interfere with neurotransmitters and are known for disrupting thyroid and steroid hormones and increasing the risk of both non-Hodgkins lymphoma and dementia, but they have also been associated with chromosome damage in sperm. Far less scientific attention has been devoted to these chemicals effects on male reproduction than on their neurological ones, but the reproductive consequences may be even greater. For one thing, many pesticides, including organophosphates, can cross the placental barrier if the mother is exposed during pregnancy. Fetal exposures to organophosphates affect childhood cognition and coordination and predispose the child to develop cancer in later life.

But it gets worse. A fathers environmental exposures can alter not only his direct fertility but also his epigenetic patterns, and these can be passed from parent to child. Epigenetics are a suite of cell processes in which gene expression is controlled by molecules that block or open access to genes in the double DNA helix. In every cell of the body, this process continually operates to orchestrate the cells biochemistry and its relation to other cells and organs, but it does not change genes themselves. Epigenetic patterns are a kind of template or history of the habits and exposures of the parent, including smoking history, diet, pesticide exposures, alcohol and drug consumption, and social stress. Sperm are major contributors of epigenetic information passed from one generation to the next, and pesticides affect that information.

It is becoming clear that epigenetic information can function as molecular memory of past environmental exposures and be passed from one generation to another via the germline, according to the authors of a 2022 review by a pair of Georgetown University Medical Center and Lombardi Comprehensive Cancer Center scholars. Descendants of an exposed male may have no direct exposure themselves but be paying for the inadvertent sins of their fatherssins such as agricultural or factory work.

A 2023 update of the 2017 review of temporal trends in sperm count, also co-authored by Swan, expanded the geographical range of the study by including data on men in 53 countries on six continents to get a global picture rather than one focused on industrialized countries where data is more plentiful. They found strong evidence that sperm counts have declined globally. Disturbingly, the authors show that the downward trend in sperm counts has become steeper since 2000, accelerating beyond the already-worrisome rate seen in the 2017 meta-analysis. From 1972 to 1999, sperm count dropped by about one percent a year; since 2000, the rate has been about 2.6 percent.

The evidence has continued to mount that pesticides affect both male and female reproductive health, yet most of these chemicals remain on the market, contributing to the prospect of agricultural collapse and declining human population worldwide. There is no longer any time to waste. What Beyond Pesticides said in 2022 still holds: As the human civilization grapples with a range of cascading crises, from climate change to the insect apocalypse and global biodiversity crisis, we may be missing the chance to address one of the most critical aspects to the continuation of humanity as we now know it.

For more information on the fertility crisis, see Dr. Swans presentation to Beyond Pesticides 2021 National Pesticide Forum, Cultivating Healthy Communities, on Beyond Pesticides YouTube page.

All unattributed positions and opinions in this piece are those of Beyond Pesticides.

Sources:

Temporal trends in sperm count: a systematic review and meta-regression analysis https://pubmed.ncbi.nlm.nih.gov/28981654/

Temporal trends in sperm count: a systematic review and meta-regression analysis of samples collected globally in the 20th and 21st centuries https://academic.oup.com/humupd/article/29/2/157/6824414?login=false

Pesticides and Male Fertility: A Dangerous Crosstalk https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8707831/

Paternal Transmission of Stressed-Induced Pathologies https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3217197/

Scientific Literature Review Again Connects Pesticides and Male Fertility Problem

Sperm counts worldwide are plummeting faster than we thought https://www.nationalgeographic.co.uk/science-and-technology/2022/11/sperm-counts-worldwide-are-plummeting-faster-than-we-thought

The Sperm-Count Crisis Doesnt Add Up https://www.nytimes.com/2021/06/04/health/sperm-fertility-reproduction-crisis.html

Read this article:
Study Confirms Connection Between Exposure to Pesticides and ... - Beyond Pesticides

Healthcare workers have the right to access the most up-to-date evidence-based information, to help them provide comprehensive abortion care. In recognition of this, the World Health Organization (WHO) andthe UN Special Programme of Research in Human Reproduction (HRP) launched a new digital decision support tool or app, available onAppleandGoogle Play, to support caregivers in the process of decision-making, and using the WHOAbortion care guideline, to provide comprehensive abortion care.

This app takes the individual characteristics of patients and generates patient-specific assessments or recommendations, which can then be given to healthcare providers to consider. The tool guides the healthcare worker through assessing abortion-seekers for possible risks, and also gives them checklists and further context to help them in managing cases. It helps to minimize possible mistakes in abortion provision, and even schedules individualized post-abortion follow-ups and referrals.

While the tool cannot be used to store client information and isnt intended for training purposes, it is likely that it will help to increase the capacity of healthcare providers working on abortion provision. This is because it will harness health workers knowledge, guiding them through clinical guidelines, combined with the clients information. It can be used as a digital job aid, on a mobile device.

This new tool recognizes the crucial importance of supporting healthcare workers to give the essential health care of abortion and post-abortion care. The app also incorporates resources relevant for the provision of abortion care including WHO guidelines, publications, evidence briefs and infographics.

Additional links:

The rest is here:
WHO Comprehensive Abortion Care Tool - UN Mobile App - United Nations - Europe News

A US appeals court has rejected a ruling that would have allowed millions of Ohio residents to sue 3M, DuPont, and others as a group over contamination by so-called toxic "forever chemicals."

A lower court had approved a massive class action, in which virtually every Ohio citizen could have sued the chemical companies due to PFAS (per- and polyfluoroalkyl substances) found in the bloodstream of the lead plaintive Kevin Hardwick.

The 6th US Circuit Court of Appeals in Cincinnati, Ohio, reversed that decision, stating very strongly that the complaint was too broad.

PFAS chemicals are widely used in many industrial applications relevant to data centers, including silicon chip manufacturing and two-phase immersion cooling. In response to findings that they can cause harm, 3M has withdrawn its Novec coolant along with other PFAS substances.

Two-phase cooling companies have moved to broaden their product lines. For instance, LiquidStack has launched a non-PFAS single-phase cooling system and ZutaCore says it will eliminate PFAS in 2026, through the use of alternative fluids.

The November 27 ruling, by District Judge Edmund A Sargus Jr, pulls no punches:

"Seldom is so ambitious a case filed on so slight a basis," said the Judge. The gravamen of Kevin Hardwicks complaint is that his bloodstream contains trace quantities of five chemicalswhich are themselves part of a family of thousands of chemicals whose usage is nearly ubiquitous in modern life.

"Hardwick does not know what companies manufactured the particular chemicals in his bloodstream; nor does he know, or indeed have much idea, whether those chemicals might someday make him sick; nor, as a result of those chemicals, does he have any sickness or symptoms now. Yet, of the thousands of companies that have manufactured chemicals of this general type over the past half-century, Hardwick has chosen to sue the ten defendants present here."

Sargus says that Hardwick's complaint rarely alleges an action by any one company, and also seeks to represent not just everyone in Ohio, but all residents of the United States. The district court had allowed this to proceed on behalf of Ohio residents.

PFAS chemicals have "innumerable" uses, including "medical devices, automotive interiors, waterproof clothing and outdoor gear, food packaging, firefighting foam, non-stick cookware, ski and car waxes, batteries, semiconductors, aviation and aerospace construction, paints and varnishes, and building materials," says the Judge.

Hardwick is a firefighter who used PFAS-based firefighting foams for 40 years. A blood test has found PFAS chemicals in his blood, but did not prove they came from the foams, nor did Hardwick know who made those chemicals, or who made the foams he used.

The result of this case may not affect the ongoing case against PFAS. The US Environmental Protection Agency (EPA) has classified some PFAS substances as hazardous substances, adding to the difficulties involved in using them, because surplus or used PFAS must be treated as hazardous waste.

EPA findings from March 2022 suggest that PFAS may affect human reproduction and development, harming the immune system and increasing the risks of some cancers. The European Chemicals Agency (ECHA) has published similar findings.

The EU has reversed earlier plans to ban PFAS, after industry groups argued that the chemicals are needed for technologies that will help reach net zero.

Go here to read the rest:
DuPont and 3M fight off giant lawsuit over PFAS "forever chemicals" - DatacenterDynamics

For those wondering what life in Palestine looks like, Condom Lead (2013), directed by Palestinian twins Arab and Tarzan Nasser, offers a striking visual metaphor: The short film opens with an apartment full of balloons, drawing the viewer in. But the scripted work takes place during the first Gaza War in 2008 and 2009. Why are there so many balloons in this house during a war, when there is no celebration occurring?

That night, we see the residents of the house, a married couple, as they try to have sex. They draw toward each other, softly touching feet and thighs, but they are interrupted by the sound of bombs, which makes their infant cry. The husband then takes a condom, blows it up, and lets it float through the apartment wherever it may land on the floor, on the bookcase, on their child. We realize this is his compulsion, a coping technique, a way of keeping score of what is taken from them.

Over the last seven weeks, life in Gaza has been quite literally unimaginable. Following the October 7 attacks by Hamas that killed 1,200 Israelis and Israels subsequent siege of Gaza with its 13,000 Palestinian deaths, there have been intermittent communications blackouts in the territory. The siege has meant Palestinians are contending with a full-blown humanitarian crisis, including attacks on refugee camps and hospitals and increased violence in the West Bank. Even knowing all that, communication failures and incredible challenges for journalists mean there is so much we dont know.

This story, however, did not begin in October 2023; the roots of the conflict reach much further back. By understanding what came before, and what everyday life looks like for people, couples, and families under occupation, we can add to our understanding of whats happening now and how we got here. A selection of short films, all easily available on Netflix, from Palestinian directors can give viewers outside the region a sense of the alienation, oppression, and human longing that have characterized life in the territories for decades. These films tell the story of trying to make a life under sustained duress.

By the end of the 15-minute Condom Lead, the apartment is even more full of balloons, representing 22 days since the couple has successfully had sex. Each balloon stands for a missed opportunity for communion, intimacy, and love. Each balloon represents an act of Israeli aggression, an occupation whose chokehold is so strong it invades even this couples bed. Were not told what this couples plans for children are, but judging by the condoms, we know theyre not looking to conceive right now. We know, at least, that their home is currently being bombed. Not only has the military assault made having children feel fraught and dangerous, but it has taken away the opportunity for closeness.

The specter of the Israeli forces looms large throughout these films, but maybe nowhere so intensely as in the Israeli prison system, the location of writer-director Rakan Mayasis Bonbon (2017). In this film, a Palestinian woman (Rana Alamuddin) smuggles sperm from her imprisoned husband (Saleh Bakri) so that she can become pregnant.

When director Mayasi, who, like many members of the Palestinian diaspora is prevented by the Israeli occupation to visit or live in Palestine, heard stories of couples navigating love and procreation amid the prison system, he felt an urge to put it in his art. The strength, beauty, and creativity of resisting occupation with love is a subject that needs to be told, he says.

The Israeli prison system is harrowing for Palestinians. The testimony of Mazen Abu Arish, a 22-year-old surveyor from the West Bank who spent 20 days in solitary confinement in Israels Shikma prison, speaks clearly to the spirit-breaking conditions; In there, you have no room to move and no desire to do a thing, he wrote.

Bonbon is set against this backdrop and addresses the issue of reproduction, both sexual and social, says Umayyah Cable, a Palestinian-American professor at the University of Michigan who researches the role that art, film, and media play in the mobilization of Palestine solidarity politics. The film speaks to anxieties and worries about Palestinian sexuality, the nuclear family, intimacy, and the literal reproduction of Palestinian society.

Israel does not allow conjugal visits for prisoners, so smuggling sperm is the only way families can reproduce when a partner is incarcerated. In 2020, Walid Daqqah, sentenced to life in prison, petitioned the Israeli court to allow him to have children with his wife San Salameh in a fertility clinic. His request was denied, so he smuggled his sperm to his wife, leading to the birth of their daughter Milad, whose name means birth in Arabic. This story inspired Mayasi. I think such a story needs to be told, the director told Short of the Week. It is so beautiful to defy occupation and resist with love and life.

Conceiving in this way has an inevitable element of dehumanization, but it also shows how Palestinians resist their oppression. Bonbon doesnt shy away from humiliation; the film shows the husband trying to masturbate as practice the night before but having trouble, his attempts constantly interrupted by sounds of prison guard announcements and metal cages clinging. Its clear that here, in this prison, he cannot connect with himself in such an intimate way. When his wife comes the next day, her body is violated by the Israeli female prison guard, who makes her strip naked, puts her hands in her hair, and forces her to bend over and squat.

The Israeli state is extremely preoccupied with Palestinian reproduction, Cable says. Demographically, Palestinians outnumber Jewish Israelis. As we know from apartheid South Africa and the Jim Crow South in the US, minority rule over a majority population is not only frowned upon by human rights agencies and the United Nations, its recognized as anti-democratic.

In 2021, an Israeli professor argued in the right-wing tabloid Israel Hayom that, Our strategy has to be demographic expansion and blocking Arab-Muslim migration to Israel. If we dont understand that victory in the conflict Jewish, or, God forbid, Arab is demographic in nature rather than military, then we will lose.

Bonbon doesnt end the story with degradation, choosing instead to give the couple moments of love and eroticism. When the wife sees her husband, she is joyous and hopeful, asking what they will name the child if he is a boy. When her husband informs her that he might have difficulty performing, she takes it upon herself to arouse him right there through the glass. Its not particularly graphic, but it is beautiful. She focuses the fantasy on a time when he was free, when they made love during a stolen moment at his brothers engagement party, when they felt connected to each other and to their community. It is hard to tell if his arousal is physical or emotional, whether he is imagining his wifes body or simply imagining being free, being allowed to connect with another human.

I generally like to deconstruct stereotypes and challenge norms, and I found Bonbon a fruitful opportunity to do that. It innately has lovemaking in it, it is never an added scene or an added tool in the film; it is the central idea the film is built around, director Mayasi tells Vox. Taking the film into the genre of sensual eroticism has given the film a louder and bolder voice. This also changed the power dynamic at the prison, the couple were stronger than their occupiers.

Despite prison conditions, the husband in Bonbon is able to feel desire and connection, even through the glass. Victorious, his wife retrieves the semen from him, smuggled in a candy wrapper (hence the title, a play on the French word for candy). On the way home, her bus is stopped by soldiers who search the bus. Once again, her attempt at a family is threatened. But she is not deterred, looking around to make sure the women are either asleep or looking away, and inseminates herself right there on the bus. It is an ending that has triumph, agency, and resilience, a portrait of a people who refuse to be denied their humanity.

As Palestinian film director Farah Nabulsi, director of The Present (2020), tells Vox, the systemic tyranny Palestinians face spreads to the realm of love and intimacy.

The pervasive stress and anxiety of living in a constant state of fear can create emotional distance and conflict in intimate relationships. Restrictions on movement and segregation policies can severely limit opportunities for meeting partners and maintaining relationships, Nabulsi says.

In The Present, Nabulsis film, a father in the West Bank named Yusuf (Saleh Bakri) and his daughter Yasmin (Maryam Kanj) set out on what seems a simple task: buying his wife and her mother Noor (Mariam Basha) an anniversary present specifically, a new refrigerator. But the labyrinth of checkpoints and violence inflicted there makes what should have been a day of bonding between a daughter and father into a traumatic experience.

When they first try to leave, the Israeli soldiers force Yusuf to wait in a holding pen with other men. He asks them not to because he is with his daughter, but his pleas only seem to make them more insistent on cruelty. Later, after he is released, he sees that Yasmin has urinated herself because the wait was so long and traumatic. When Yusuf expresses concern and tells her she should have spoken up, Yasmin says, Its okay, Dad. There was nothing you could do. His face crumples upon hearing this. A parents job is to protect their child, and he is devastated to see that at such a young age, she is already learning that, in the occupation, there are limits to what her father can do to protect her.

Nabulsi tells Vox that this story highlights how the occupation seeps its way into the fabric of family life for Palestinians. In this hardship, the roots of their bond might grow deeper. The shared ordeal becomes a silent teacher of empathy. The young girl may come to understand the depth of her fathers struggles and the complexities of the world they navigate.

Its demonstrated to both of them again, at night, as they attempt to roll the fridge past the checkpoint. Even though their house is right there, in sight, the Israeli soldiers order them to take an hours-long detour. The soldiers dehumanize the family further, searching their grocery bags to find Yasmins soiled pants from before and taunting them. Youre all disgusting, one of the Israeli soldiers spits.

Yusuf pleads until he demands forcefully to be let through, resorting to yelling and banging on the table. Its a terrifying moment: The Israeli soldiers guns are pointed at him, and the audience imagines how this will end a father shot to death in front of his daughter but then we hear a creaking of the gate and see Yasmin, looking smaller than she has looked the entire film but somehow also stronger, rolling the refrigerator past the checkpoint herself. Yusuf and the soldiers are stunned, and Yusuf begins to walk alongside his daughter, who resolutely keeps going. It is a deeply sad triumph. And as Nabulsi points out, it is ultimately unrealistic.

The stark reality often dictates a grim outcome either an encounter with deadly force or the infliction of physical injury and/or arrest. But as a storyteller often drawn to the somber hues of human experience, I felt compelled to offer an ending with more hope, Nabulsi says. A suggestion that hinted at a brighter future, spearheaded by the youth interestingly, a female. Its her, and other youth like her, emerging resilient and assertive, who captivate my imagination.

I remain a woman anchored by hope, by an unwavering faith in the strength and potential of my community, Nabulsi continues. This film is a testament to that belief: a narrative that ultimately chooses to embrace the possibility of change and the promise of a generation poised to redefine their destiny.

Will you support Voxs explanatory journalism?

Most news outlets make their money through advertising or subscriptions. But when it comes to what were trying to do at Vox, there are a couple of big issues with relying on ads and subscriptions to keep the lights on:

First, advertising dollars go up and down with the economy. We often only know a few months out what our advertising revenue will be, which makes it hard to plan ahead.

Second, were not in the subscriptions business. Vox is here to help everyone understand the complex issues shaping the world not just the people who can afford to pay for a subscription. We believe thats an important part of building a more equal society. And we cant do that if we have a paywall.

So even though advertising is still our biggest source of revenue, we also seek grants and reader support. (And no matter how our work is funded, we have strict guidelines on editorial independence.)

If you also believe that everyone deserves access to trusted high-quality information, will you make a gift to Vox today? Any amount helps.

Yes, I'll give $5/month

Yes, I'll give $5/month

We accept credit card, Apple Pay, and Google Pay. You can also contribute via

Read the original:
Review: These 3 Netflix short films provide some insight to life in ... - Vox.com

GoalCycle3D task space

We introduce GoalCycle3D, a 3D physical simulated task space built in Unity38,39 which expands on the GoalCycle gridworld environment of ref. 33. By anchoring our task dynamics to this previous literature and translating it to a 3D space, our results naturally extend prior work to a more naturalistic and realistic environment. The resulting richness is an important direction for the eventual deployment of AI, highlighting which algorithmic novelties are required to exceed the prior state-of-the-art in a more realistic setting.

Similar to ref. 27, we decompose an agents task as the direct product of a world, a game and a set of co-players. The world comprises the size and topography of the terrain and the locations of objects. The game defines the reward dynamics for each player, which in GoalCycle3D amounts to a correct ordering of goals. A co-player is another interactive policy in the world, consuming observations and producing actions. Each task can be viewed as a different Markov decision process, thus presenting a distribution of environments for reinforcement learning.

While the 3D task space yields significant richness, it also presents opportunities for handcrafting which would reduce the generality of our findings. To avoid this, we make use of procedural generation over a wide task space. More specifically, we generate worlds and games uniformly at random for training, and test generalisation to held-out probe tasks at evaluation time, including a held-out human co-player, as described in Probe Tasks. This train-test split provides data that enables overfitting to be ruled out, just as in supervised learning.

Worlds are parameterised by world size, terrain bumpiness and obstacle density. The obstacles and terrain create navigational and perception challenges for players. Players are positively rewarded for visiting goal spheres in particular cyclic orders. To construct a game, given a number of goals n, an order Sn is sampled uniformly at random. The positively rewarding orders for the game are then fixed to be {, 1} where 1 is the opposite direction of the order . An agent has a chance (frac{2}{(n-1)!}) of selecting a correct order at random at the start of each episode. In all our training and evaluation we use n4, so one is always more likely to guess incorrectly. The positions and orders of the goal spheres are randomly sampled at the start of each episode.

Players receive a reward of +1 for entering a goal in the correct order, given the previous goals entered. The first goal entered in an episode always confers a reward of +1. If a player enters an incorrect goal, they receive a reward of 1 and must now continue as if this were the first goal they had entered. If a player re-enters the last goal they left, they receive a reward of 0. The optimal policy is to divine a correct order, by experimentation or observation of an expert, and then visit the spheres in this cyclic order for the rest of the episode. Figure1 summarises the GoalCycle3D task space.

A 3D physical simulated task space.Each task contains procedurally generated terrain, obstacles, and goal spheres, with parameters randomly sampled on task creation. Each agent is independently rewarded for visiting goals in a particular cyclic order, also randomly sampled on task creation. The correct order is not provided to the agent, so an agent must deduce the rewarding order either by experimentation or via cultural transmission from an expert. Our task space presents navigational challenges of open-ended complexity, parameterised by world size, obstacle density, terrain bumpiness and a number of goals. Our agent observes the world using LIDAR (see Supplementary Movie30).

The term cultural transmission has a variety of definitions, reflecting the diverse literature on the subject. For the purpose of clarity, we adopt a specific definition in this paper, one that captures the key features of few-shot imitation. Intuitively, the agent must improve its performance upon witnessing an expert demonstration and maintain that improvement within the same episode once the demonstrator has departed. However, what seems like test-time cultural transmission might actually be cultural transmission during training, leading to memorisation of fixed navigation routes. To address this, we measure cultural transmission in held-out test tasks and with human expert demonstrators40,41, similar to the familiar train-test dataset split in supervised learning42.

Capturing this intuition, we define cultural transmission from expert to agent to be the average of improvement in agent score when an expert is present and improvement in agent score when that expert has subsequently departed, normalised by the expert score, evaluated on held-out tasks that have never before been experienced by the agent. Mathematically, let E be the total score achieved by the expert in an episode of a held-out task. Let Afull be the score of an agent with the expert present for the full episode. Let Asolo be the score of the same agent without the expert. Finally, let Ahalf be the score of the agent with the expert present from the start to halfway into the episode. Our metric of cultural transmission is

$${{{{{{{rm{CT}}}}}}}}:!!!=frac{1}{2}frac{{A}_{{{{{{{{rm{full}}}}}}}}}-{A}_{{{{{{{{rm{solo}}}}}}}}}}{E}+frac{1}{2}frac{{A}_{{{{{{{{rm{half}}}}}}}}}-{A}_{{{{{{{{rm{solo}}}}}}}}}}{E},.$$

(1)

A completely independent agent doesnt use any information from the expert. Therefore it has a value of CT near 0. A fully expert-dependent agent has a value of CT near 0.75. An agent that follows perfectly when the expert is present, but continues to achieve high scores once the expert is absent has a value of CT near 1. This is the desired behaviour of an agent from a cultural transmission perspective, since the knowledge about how to solve the task was transmitted to, retained by and reproduced by the agent.

We first examine how reinforcement learning can generate cultural transmission in a relatively simple setting, a 4-goal game in a 2020m2 empty world. This is far from the most challenging task space for our algorithm, but it has a simplicity that is useful for developing our intuition. We find that an agent trained with memory (M), expert dropout (ED), and an attention loss (AL) on tasks sampled in this subspace experiences 4 distinct phases of training. The learning pathway of the agent passes through a cultural transmission phase to reach a policy that is capable of online adaptation, experimenting to discover and exploit the correct cycle within a single episode. By comparison, a vanilla RL baseline (M) is incapable of learning this few-shot adaptation behaviour. In fact it completely fails to get any score on the task (see The role of memory, expert demonstrations and attention loss). Cultural transmission, then, is functioning as a bridge to few-shot adaptation.

The training cultural transmission metric shows four distinct phases over the training run, each corresponding to a distinct social learning behaviour of the agent (see Fig.2). In phase 1 (red), the agent starts to familiarise itself with the task, learns representations, locomotion, and explores, without much improvement in score. In phase 2 (blue), with sufficient experience and representations shaped by the attention loss, the agent learns its first social learning skill of following the expert bot to solve the task. The training cultural transmission metric increases to 0.75, which suggests pure following.

Training cultural transmission (left) and agent score (right) for training without ADR on 4-goal in a small empty world. Colours indicate four distinct phases of agent behaviour from left to right: (1) (red) startup and exploration, (2) (blue) learning to follow, (3) (yellow) learning to remember, (4) (purple) becoming independent from expert.

In phase 3 (yellow), the agent learns the more advanced social learning skill that we call cultural transmission. It remembers the rewarding cycle while the expert bot is present and retrieves that information to continue to solve the task when the bot is absent. This is evident in a training cultural transmission metric approaching 1 and a continued increase in agent score.

Lastly, in phase 4 (purple), the agent is able to solve the task independent of the expert bot. This is indicated by the training cultural transmission metric falling back towards 0 while the score continues to increase. The agent has learned a memory-based policy that can achieve high scores with or without the bot present. More precisely, MEDAL displays an experimentation behaviour in this phase, which involves using hypothesis-testing to infer the correct cycle without reference to the bot, followed by exploiting that correct cycle more efficiently than the bot does (see Supplementary Movies14). The bot is not quite optimal because for ease of programming it is hard-coded to pass through the centre of each correct goal sphere, whereas reward can be accrued by simply touching the sphere. Note by comparison with Fig.3a that this experimentation behaviour does not emerge in the absence of prior social learning abilities.

Score (left), training cultural transmission (CT, centre), and evaluation CT on empty world 5-goal probe tasks (right) over the course of training. a Comparing MEDAL with three ablated agents, each trained without one crucial ingredient: without an expert (M), memory (EDAL), or attention loss (MED). b Ablating the effect of expert dropout, comparing no dropout (MEAL) with expert dropout (MEDAL). We report the mean performance for each across 10 initialisation seeds for agent parameters and task procedural generation. We also include the experts score and MEDALs best seed for scale and upper-bound comparisons. The shaded area on the graphs is one standard deviation.

In other words, few-shot imitation creates the right prior for few-shot adaptation to emerge, which remarkably leads to improvement over the original demonstrators policy. Note that, social learning by itself is not enough to generate experimentation automatically, further innovation by reinforcement learning, on top of the culturally transmitted prior, is necessary for the agent to exceed the capabilities of its expert partner. Our agent stands on the shoulders of giants, and then riffs to climb yet higher.

We have shown that our MEDAL agent is capable of learning a test-time cultural transmission ability. Now, we show that the set of ingredients is minimal, by demonstrating the absence of cultural transmission when any one of them is removed. In every experiment, MEDAL and its ablated cousins were trained on procedurally generated 5-goal, 2020 worlds with no vertical obstacles and horizontal obstacles of density 0.0001m2, and evaluated on the empty world 5-goal probes in Probe tasks. We use a variety of different dropout schemes, depending on the ablation. M- is trained with full dropout (expert is never present), MEAL is trained with no dropout (expert is always present) and all other agents are trained with probabilistic dropout.

Figure3a shows that memory (M), the presence of an expert (E), and our attention loss (AL) are important ingredients for the learning of cultural transmission. In the absence of these the agent achieves 0 score and therefore also doesnt pick up any reward-influencing social cues from the expert (if present), accounting for a mean CT of 0.

First, we consider the M- ablation. By removing expert demonstrations and, consequently, all dependent components, the dropout (D) and attention loss (AL), the agent must learn to determine the correct goal ordering by itself in every episode. The MPO agents exploration strategy is not sufficiently structured to deduce the underlying conceptual structure of the task space, so the agent simply learns a risk-averse behaviour of avoiding goal spheres altogether (see Supplementary Movie5).

Next, we analyse the EDAL ablation. Without memory, our agent cannot form connections to previously seen cues, be they social, behavioural, or environmental. When replacing the LSTM with an equally sized MLP (keeping the same activation functions and biases, but removing any recurrent connections), our agents ability to register and remember a solution is reduced to zero.

Lastly, we turn to the MED ablation. Having an expert at hand is futile if the agent cannot recognise and pay attention to it. When we turn off the attention loss, the resulting agent treats other agents as noisy background information, attempting to learn as if it were alone. Vanilla reinforcement learning benefits from social cues to bootstrap knowledge about the task structure; the attention loss encourages it to recognise social cues. Note that the attention loss, like all auxiliary losses to shape neural representations, is only required at training time. This means that our agent can be deployed with no privileged sensory information at test time, relying solely on its LIDAR.

To isolate the importance of expert dropout, we compare our MEDAL agent (in which the expert intermittently drops in and out) with the previous state-of-the-art method ME-AL (in which the expert is always present). We use the same procedural generation and evaluation setting as in the previous section. Studying Fig.3b, we see that the addition of expert dropout to the previous state of the art leads to better CT. MEDAL achieves higher CT both during training and when evaluated on empty world 5-goal probe tasks. This is because dropout encourages the learning of within-episode memorisation, a capability that was absent from previous agents33 and which confers a higher cultural transmission score (see also Agents recall expert demonstrations with high fidelity).

As we have seen, learning cultural transmission in a fixed task distribution acts as a gateway for learning few-shot adaptation. While this is undeniably useful in its own right, it begs the question: how can an agent learn to transmit cultural information in more complex tasks? ADR is a method of expanding the task distribution across training time to maintain it in the Goldilocks zone for cultural transmission. It gradually increases the complexity of the training worlds in an open-ended procedurally generated space (parameterised by 7 hyperparameters).

Figure4a shows an example expansion of the randomisation ranges for all parameters for the duration of an experiment. Training CT is maintained between the boundary update thresholds 0.75 and 0.85. We see an initial start-up phase of ~100 hours when social learning first emerges in a small, simple set of tasks. Once training CT exceeds 0.75, all randomisation ranges began to expand. Different parameters expand at different times, indicating when the agent has mastered different skills such as jumping over horizontal obstacles or navigating bumpy terrain. For intuition about the meaning of the parameter values, see Supplementary Movies69.

a The expansion of parameter ranges over training for one representative seed in MEDAL-ADR training. b Score (left), training Cultural Transmission (CT, centre), and evaluation CT on complex world probe tasks (right) over the course of training for the automatic (A) and domain randomisation (DR) ablations of MEDAL-ADR. We report the mean performance for each across 10 initialisation seeds for agent parameters and task procedural generation. We also include the experts score and the best MEDAL-ADR seed for scale and upper bound comparisons. The shaded area on the graphs is one standard deviation.

To understand the importance of ADR for generating cultural transmission in complex worlds, we ablate the automatic (A) and domain randomisation (DR) components of MEDAL-ADR (for parameter values, see Supplementary TableD.1). The MEDAL agent is trained on worlds as complicated as the end point of the ADR curriculum. The MEDAL-DR agent is trained on a uniformly sampled distribution between the minimal and maximal complexities of the ADR curriculum (i.e., no automatic adaptation of the curriculum). In Fig.4b we observe that ADR is crucial for the generation of cultural transmission in complex worlds, with MEDAL-ADR achieving significantly higher scores and cultural transmission than both MEDAL-DR and MEDAL.

To demonstrate the recall capabilities of our best-performing agent, we quantify its performance across a set of tasks where the expert drops out. The intuition here is that if our agent is able to recall information well, then its score will remain high for many timesteps even after the expert has dropped out. However, if the agent is simply following the expert or has poor recall, then its score will instead drop immediately close to zero. To our knowledge, within-episode recall of a third-person demonstration has not previously been shown to arise from reinforcement learning. This is an important discovery, since the recent history of AI research has demonstrated the increased flexibility and generality of learned behaviours over pre-programmed ones. Whats more, third-person recall within an episode amortises imitation onto a timescale of seconds and does not require perspective matching between co-players. As such, we achieve the fast adaptation benefits of previous first-person few-shot imitation works (e.g., refs. 22,43,44) but as a general-purpose emergent property from third-person RL rather than via a special-purpose first-person imitation algorithm.

For each task, we evaluate the score of the agent across ten contiguous 900-step trials, comprising an episode of experience for the agent. In the first trial, the expert is present alongside the agent, and thus the agent can infer the optimal path from the expert. From the next trial onwards the expert is dropped out and therefore the agent must continue to solve the task alone. The world, agent, and game are not reset between trial boundaries; we use the term trial to refer to the bucketing of score accumulated by each player within the time window. We consider recall from two different experts, a scripted bot and a human player. For both, we use the worlds from the 4-goal probe tasks (see Automatic domain randomisation).

Figure5 compares the recall abilities of our agent trained with expert dropout (MEDAL-ADR) and without (ME-AL, similar to the prior state of the art33). Notably, after the expert has dropped out, we see that our MEDAL-ADR agent is able to continue solving the task for the first trial while the ablated ME-AL agent cannot. MEDAL-ADR maintains a good performance for several trials after the expert has dropped out, despite the fact that the agent only experienced 1800-step episodes during training. From this, we conclude that our agent exhibits strong within-episode recall.

Score of MEDAL-ADR and ME-AL agents across trials since the expert dropped out. a Experts are scripted bots. b Experts are human trajectories. Supplementary Movie10 shows MEDAL-ADRs recall from a bot demonstration in a 3600-step (4 trial) episode. Supplementary Movie31 shows MEDAL-ADRs recall from a human demonstration in an 1800-step (2 trial) episode.

To show causal information transfer from the expert to the agent in real time, we can adopt a standard method from the social learning literature. In the two-action task28,29,30 subjects are required to solve a task with two alternative solutions. Half of the subjects observe a demonstration of one solution while the others observe a demonstration of the alternative solution. If subjects disproportionately use the observed solution, this is evidence that supports imitation. This experimental approach is widely used in the field of social learning; we use it here as a behavioural analysis tool for artificial agents for the first time. Using the tasks from our game space analysis, we record the preference of the agent in pairs of episodes where the expert demonstrates the optimal cycles and 1. The preference is computed as the percentage of correct complete cycles that an agent completes that match the direction of the expert cycle. Evaluating this over 1000 trials, we find that the agents preference matched the demonstrated option 100% of the time, i.e., in every completed cycle of every one of the 1000 trials.

Trajectory plots further reveal the correlation between expert and agent behaviour (see Fig.6). By comparing trajectories under different conditions, we can again argue that cultural transmission of information from expert to agent is causal. The agent cannot solve the task when the bot is not placed in the environment (Fig.6a). When the bot is placed in the environment, the agent is able to successfully reach each goal and then continue executing the demonstrated trajectory after the bot drops out (Fig.6b). However, if an incorrect trajectory is shown by the expert, the agent still continues to execute the wrong trajectory (Fig.6c).

Trajectory plots for MEDAL-ADR agent for a single episode. a The bot is absent for the whole episode. b The bot shows a correct trajectory in the first half of the episode and then drops out. c The bot shows an incorrect trajectory in the first half of the episode and then drops out. The coloured parts of the lines correspond to the colour of the goal sphere the agent and expert have entered and thes correspond to when the agent entered the incorrect goal. Here, position refers to the agents position along the z-axis. Supplementary Movies1113 correspond to each plot respectively.

To demonstrate the generalisation capabilities of our agents, we quantify their performance over a distribution of procedurally generated tasks, varying the underlying physical world and the overlying goalcycle game. We analyse both in-distribution and out-of-distribution generalisation, with respect to the distribution of parameters seen in training (see Supplementary TableC.2). Out-of-distribution values are calculated as 20% of the min/max in-distribution ADR values where possible, and indicated by cross-hatched bars in all figures.

In every task, an expert bot is present for the first 900 steps, and is dropped out for the remaining 900 steps. We define the normalised score as the agents score in 1800 steps divided by the experts score in 900 steps. An agent who can perfectly follow but cannot remember will score 1. An agent which can perfectly follow and can perfectly remember will score 2. Values in between correspond to increasing levels of cultural transmission.

The space of worlds is parameterised by the size and bumpiness of the terrain (terrain complexity) and the density of obstacles (obstacle complexity). To quantify generalisation over each parameter in this space, we generate tasks with worlds sampled uniformly from the chosen parameter while setting the other parameters at their lowest in-distribution value. Games are then uniformly sampled across the possible number of crossings for 5 goals.

From Fig.7a, we conclude that MEDAL-ADR generalises well across the space of worlds, demonstrating both following and remembering across the majority of the parameter variations considered, including when the world is out-of-distribution.

a A slice through the world space allows us to disentangle MEDAL-ADRs generalisation capability across different world space parameters. b MEDAL-ADR generalises across the game space, demonstrating remembering capability both inside and outside the training distribution. We report the mean performance across 50 initialisation seeds for a and 20 initialisation seeds for b. The error bars on the graphs represent 95% confidence intervals. Supplementary Movies1420 demonstrate generalisation over the world space and game space.

The space of games is defined by the number of goals in the world as well as the number of crossings contained in the correct navigation path between them. To quantify generalisation over this space, we generate tasks across the range of feasible N-goal M-crossing games in a flat empty world.

Figure7b shows our agents ability to generalise across games, including those outside of its training distribution. Notably, MEDAL-ADR can perfectly remember all numbers of crossings for the in-distribution 5-goal game. We also see impressive out-of-distribution generalisation, with our agent exhibiting strong remembering, both in 4-goal and 0-crossing 6-goal games. Even in complex 6-goal games with many crossings, our agent can still perfectly follow.

Deep learning models are not necessarily readily interpretable. On the other hand, interpretability is often desirable or even pre-requisite for deploying AI systems in the real world. Here, we demonstrate that our model is interpretable at the neural level. Training agents to imitate via meta-reinforcement learning embeds the logic for a state-machine capable of approximately Bayes-optimal cultural transmission into the neural networks weights45. By inspecting a trained agents memory, we find clearly interpretable individual neurons. These neurons have specialised roles required for solving a new task online via cultural transmission, a subset of the sufficient statistics which drive the state-machine46. One, dubbed the social neuron, encapsulates the notion of agency; the other, called the goal neuron, captures the periodicity of the task.

To identify the social neuron, we use linear probing47,48, a well-known and powerful method for understanding intermediate layers of a deep neural network. We train an attention-based classifier to classify the presence or absence of an expert co-player based on the memory state of the agent. The neuron with the maximum attention weight is defined to be a social neuron, and its activation crisply encodes the presence or absence of the expert in the world (Fig.8a). Figure8b shows a stark difference in prediction accuracy for expert presence between differently ablated agents. This suggests that the attention loss (AL) is at least partly responsible for incentivising the construction of socially-aware representations.

a Activations for MEDAL-ADRs social neuron. b We report the accuracy of three linear probing models trained to predict the experts presence based on the belief states of three agents (MED, MEDAL, and MEDAL-ADR). We make two causal interventions (in green and purple) and a control check (in red) on the original test set (yellow). We report the mean performance across 10 different initialisation seeds. The small standard deviation error bars suggest a broad consensus across the 10 runs on which neurons encode social information. c Spikes in the goal neurons activations correlate with the time the agent remains inside a goal (illustrated by coloured shading). The goal neuron was identified using a variance analysis, rather than the linear probing method in b.

To identify the goal neuron we inspect the variance of memory neural activations across an episode, finding a neuron whose activation is highly correlated with the entry of an agent into a goal sphere. Figure8c shows that this neuron fires when the agent enters and remains within a goal sphere. Interestingly, it is not the presence or the following of an expert that determines the spikes, nor the observation of a positive reward. Appendix D.3 contains full details of our methods and results.

More:
Learning few-shot imitation as cultural transmission - Nature.com

Researchers at the Ohio State University collected 9,287 Asian longhorned ticks in just 90 minutes using lint rollers. Risa Pesapane via Ohio State

An invasive species of tick that can clone itself has been spreading rapidly across the eastern United Statesand now, researchers have documented a population that killed three cows on an Ohio farm.

This marks the first established population of this species, called the Asian longhorned tick, in the state, according to a paper recently published in the Journal of Medical Entomology.The ticks pose a serious threat to livestock, because they congregate in the thousands and can drain an entire animal of blood.

The tick will be a nuisance, and it is spreading, Kevin Lahmers, an anatomic pathologist at the Virginia-Maryland College of Veterinary Medicine who was not involved with the study, tells Jenny McGrath of Business Insider. It will cover most of the eastern half of the U.S.thats most likely.

Asian longhorned ticks (Haemaphysalis longicornis) are native to eastern China, Japan, the Russian Far East and Korea, but they were first documented in the U.S. on a New Jersey sheep farm in 2017. In the past six years, the arachnids have spread across 19 states, colonizing new areas incredibly quickly thanks to an unusual reproductive strategy called parthenogenesis. This mode of asexual reproduction allows females to lay about 2,000 fertile eggs without mating.

There are no other ticks in North America that do that. So they can just march on, with exponential growth, without any limitation of having to find a mate, Risa Pesapane, a disease ecology researcher at the Ohio State University, says in a statement. Where the habitat is ideal, and anecdotally it seems that unmowed pastures are an ideal location, theres little stopping them from generating these huge numbers.

In 2021, Pesapane received a call from a farmer in eastern Ohio, who reported that three of his 18 cattle had died after being infested heavily with ticks, per the statement. Pesapane and her colleagues visited the property, and in just 90 minutes, they collected more than 9,000 Asian longhorned ticks with muslin cloth and lint rollers. This massive quantity led them to believe the 25-acre property hosted more than one million of the ticks in total.

Anecdotally, Goudarz Molaei, the Connecticut Agricultural Experiment Stations chief scientist and head of entomology, experienced another large infestation in Connecticut, reports CT Insiders Vincent Gabrielle.

There was once incidence in Bridgeport after I walked out of a tick-infested area that I was able to collect 800 ticks just from my coveralls, Molaei, who was not involved with the recent study, tells the publication.

Though Asian longhorned ticks can carry diseases that infect humans, they are not yet considered a threat to human health in the U.S., per the statement. The parasites dont seem to be as attracted to human skin as other species of native ticks are. They also appear unlikely to pass on Lyme disease, though, in lab settings, they have been found to transmit other diseases of concern, including Rocky Mountain spotted fever, Heartland virus and Powassan virus, per Business Insider.

After testing 100 ticks sampled from the Ohio farm, the researchers discovered that eight were positive for Anaplasma phagocytophilum, a bacterium that can cause the disease human granulocytic anaplasmosis (HGA). Patients with HGA can experience symptoms such as fever, chills, severe headaches and nausea, but the fatality rate is very lowless than 1 percent. No other diseases were found in any of the other collected ticks.

The researchers are now working on filling in the gaps in knowledge about this tick species and coming up with better management strategies. While pesticides can kill Asian longhorned ticks, the arachnids can easily escape applications by hiding in vegetation, per the statement.

It would be wisest to target them early in the season when adults become active, before they lay eggs, because then you would limit how many will hatch and reproduce in subsequent years, Pesapane says in the statement. But for a variety of reasons, I tell people you cannot spray your way out of an Asian longhorned tick infestationit will require an integrated approach.

Get the latest stories in your inbox every weekday.

Recommended Videos

See the original post:
An Invasive Tick That Can Clone Itself Is Spreading Across the U.S. ... - Smithsonian Magazine