PhD Research Fellow – Benchmarking, Evaluating, and Improving Trustworthy Language Models
Deadline: 29.09.2025
Publisert
The University of Agder has more than 1500 employees and almost 14 000 students, making us one of Southern Norway's largest workplaces. Our dedicated staff engage in research, teaching and dissemination across a diverse range of fields.
The university is located on two modern campuses in Kristiansand and Grimstad.
PhD Research Fellow in ICT – Benchmarking, Evaluating, and Improving Trustworthy Language Models
About the position
The University of Agder invites applications for a full-time fixed-term appointment as PhD candidate in Technology/Information and Communication Technology for a period of three years with a possible extension to four years. The starting date is as soon as possible or to be negotiated with the Faculty. The position is located in Grimstad, Norway, but the candidate is expected to spend substantial time with research partners, both FHI in Oslo, Norway, and Norkart in Kristiansand, Norway.
The department has three large and active research groups in Information and Communication Technology including eight Professors, 20 Associate Professors/Assistant professors and about 30 Research Fellows on the PhD programme in ICT.
The Department of ICT pursues a variety of research interests, focusing especially on ICT, artificial intelligence, eLearning and eHealth. The department has successfully led a number of large research projects funded by the Research Council of Norway, the EU research programmes FP7 and H2020 as well as national and international industries.
The project will be conducted in close collaboration with the Norwegian Institute of Public Health (FHI), Norkart, Kristiansand Municipality, and the Norwegian Mapping Authority (Kartverket).
Research topics
The candidate will investigate and develop state‑of‑the‑art methods for benchmarking and evaluating language models in highly sensitive and complex application areas, with a focus on both the public healthcare sector and public case handling. The research aims to advance understanding of how large language models (LLMs) and other advanced language technologies can be reliably leveraged in domains where precision, accountability, and trust are critical. An integral part of the project is the practical application of LLM-based solutions in real-world settings. At partner organizations, LLM-based systems are planned to be deployed and evaluated in real environments, allowing the candidate to work with domain experts and help refine tools used in the public sector.
This Ph.D. position is associated with the Centre for Artificial Intelligence Research at the University of Agder and is expected to collaborate closely with partners in both clinical and public service settings.
The candidate will develop and investigate advanced methods and frameworks for:
Adapting and developing solutions for high-stakes applications in the public health care sector and in public case handling, ensuring their alignment with domain needs and regulatory requirements.
Developing benchmarking pipelines for language models, focusing on performance, reliability, and precision, including methods for accurate and trustworthy summarization of scientific literature, retrieval-augmented generation from structured and unstructured knowledge sources, context‑aware generation, and evaluation of reasoning accuracy, factual consistency, and faithfulness in complex scientific and public sector contexts.
Designing and applying evaluation metrics that assess the precision, factuality, and trustworthiness of language models, including their ability to reliably locate, interpret, and reproduce information from complex documents such as research papers, public health information, and case files
Potential research topics for the Ph.D. project are:
Adaptation of language model architectures and pipelines for high-stakes public sector
Benchmark and comparative analysis methods for language models, focusing on performance, precision, reliability, and explainability
Evaluation methodologies that combine qualitative expert review with quantitative performance metrics
Evaluating and testing language models in real-life public sector settings, benefiting from access to domain experts, real data, and opportunities for real-world deployment with a focus on precision, reliability, and explainability
Creating frameworks for aligning large language models with institutional policies and legal requirements, ensuring trustworthy and compliant deployments across sectors.
Ideal candidates will have expertise in the following areas:
Machine Learning in general, with an emphasis on deep learning and language modeling
Model benchmarking and evaluation pipelines for NLP/LLMs
Domain-aware application of AI methods (e.g., health informatics, legal tech)
Strong programming and experimental design abilities
An understanding of privacy, data protection, and AI ethics in highly regulated domains
Required qualifications
Applications from applicants who already hold a PhD will normally not be considered.
The candidate must hold a master’s degree in ICT, Computer Science or a similar/related field and document competence in areas relevant to the position.
Strong academic credentials
Written and spoken English proficiency. International candidates that are not exempt from the English language requirements pursuant to the guidelines of the Norwegian Agency for Quality Assurance in Education (NOKUT) must document this through one of the following tests with the stated results or better:
TOEFL - Test of English as a Foreign Language with a minimum score of 600 for the Paperbased Test (PBT), or 92 for the Internet-based Test (iBT)
IELTS - International English Language Testing System, with the result of 6.5
The PhD Research Fellow must be admitted to the PhD Programme in Technology, with ICT as specialization, within three months of appointment. More information about the programme and a complete list of admission requirements can be found here.
The following admission requirements apply to the PhD programme:
The average grade for courses included in the bachelor's degree (or equivalent) must be C (or equivalent) or higher;
The average grade for courses included in the master's degree (or equivalent) must be B (or equivalent) or higher;
The master's thesis (or equivalent) must have a grade B (or equivalent) or higher.
Desired qualifications
Communication and teamwork skills.
Written and spoken Norwegian proficiency
Personal qualities
Personal suitability, good teamwork skills, inventiveness, and a proactive approach, will be emphasized in the evaluation as well as relevant practical experience.
The PhD Research Fellow will be expected to contribute to an active research community that promotes the personal and professional growth of the doctoral candidate. The position places great demands on the applicant’s capacity for independent goal-oriented work, ability to concentrate and attention to detail.
In return, we offer the opportunity to contribute to the strategic capabilities of a world-class research organization, along with intensive PhD supervision. You will collaborate with top scientists in your field and have excellent prospects for personal development in an innovative working environment for aspiring researchers. Applicants will be assessed on the basis of academic background and results, attainment, and any previous research and development work.
We offer
professional development in a large, exciting and socially influential organization
a positive, inclusive and diverse working environment
modern facilities and a comprehensive set of welfare offers
The position is remunerated according to the State Salary Scale, salary plan 17.515, code 1017 PhD Research Fellow, NOK 550 800 gross salary per year. A compulsory pension contribution to the Norwegian Public Service Pension Fund is deducted from the pay according to current statutory provisions.
General information
UiA is an open and inclusive university. We believe that diversity enriches the workplace and makes us better. We, therefore, encourage qualified candidates to apply for the position independent of gender, age, cultural background, disability or an incomplete CV.
Women are especially encouraged to apply.
The successful applicant will have rights and obligations in accordance with the current regulations for the position, and organisational changes and changes in the duties and responsibilities of the position must be expected. The engagement is to be made in accordance with the regulations in force concerning the acts relating to Control of Export of Strategic Goods, Services and Technology. Appointment is made by the University of Agder’s Appointments Committee for Teaching and Research Positions. Appointment is made by the University of Agder’s Appointments Committee for Teaching and Research Positions.
Short-listed applicants will be invited for interview. With the applicant’s permission, UiA will also conduct a reference check before appointment. Read more about the employment process.
In accordance with the Freedom of Information Act § 25 (2), applicants may request that they are not identified in the open list of applicants. The University, however, reserves the right to publish the names of applicants. Applicants will be advised of the University’s intention to exercise this right.
Application
Submit your application and CV online. Please click on the link “Apply for this job”.
The following documentation should be submitted as attachments to the online application:
Certificates and/or grades for all post-secondary education, up to and including the bachelor's level;
Master's degree/higher degree certificate, with a summary of the courses/subjects included in the degree;
Applicants with a foreign higher education must attach an official description of the grading system used at the issuing institution;
Summary (approximately 1-2 pages) of the master's thesis;
Applicants who are required to document their English proficiency must submit their TOEFL or IELTS test results (these may be forwarded after the closing date);
Summary or links to the applicant's scientific publications (if any);
A description of the candidate’s research interests and background for the project applied for.
Original documents must be presented for verification to the University of Agder together with the copies by the appointed research fellow upon arrival at UiA. Successful candidates will be asked, normally during the interview, to ensure that the issuing university submits documents in a sealed envelope directly to UiA or provide access to their documents online, which allows UiA to verify the authenticity of these electronic documents via a secure website hosted at the issuing university (contact person at UiA will be provided later for certain candidates).
The applicant is fully responsible for submitting complete documentation within the deadline.
All documentation of education must be in the original language and in English, and the translation should preferably be from the issuing university. Additional documentation must be in English.
