In this study ChatGPT is embedded as pedagogical agent in the learning designs of two Case studies in Higher Education, involving 20 and 12 pre-service teachers respectively, to scaffold their creative and critical thinking. Thematic analysis of texts that were produced through reflective practices showed that students do not afford adequate competencies to distinguishing human from ChatGPT’s creativity and to use critical thinking to evaluate its output for further use. Findings are not generalizable but raise awareness on necessary competencies when interacting with pedagogical agents.